{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ICESat-2 Hackweek 2020: Machine Learning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Yara Mohajerani ([ymohajer@uci.edu](mailto:ymohajer@uci.edu))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note** Before going through this tutorial, make sure you have the correction environment by running\n",
    "\n",
    "`conda create --name <env> --file mlenv.lock`\n",
    "with the `mlenv.lock` file being in this repositoriy, and choose the this environment for your kernel.\n",
    "\n",
    "---\n",
    "\n",
    "In this tutorial we will explore the basics of machine learning with an emphasize on neural networks and applications in altimetry. First we will offer a brief introduction to theory and techniques in implementing neural networks, and then will focus on altimetry applications.\n",
    "\n",
    "## Motivation\n",
    "Advances in machine learning have made it a valuable tool for extracting insights from large datasets without the need for developing exact analytical algorithms. In particular, this has proven extremely useful for altimetry applications. For example, imagine you have thousands of satellite images and you want to identify paricular features automatically in the dataset. One way is to manually go through the data, which is not practical. The second way is to come up with an analytical algorithm that uses explicit engineered laws to detect the desired features. This is not very easy or robust, to say the least. But machine learning methods can learn to perform the desired task from the data in an approximate manner. Specifically, *supervised* methods use the data along with the desired outputs to accomplish the learning task. Here we will focus on supervised machine learning. An extremely useful subset of supervised machine learning fields is *neural networks*, which will be discussed in this notebook.\n",
    "\n",
    "Given the shear volume of altimetry data from ICESat-2, the is huge potential in extracting insights from the data with machine learning that were not possible before. Keep in mind that machine learning is a big field and here we focus on only a few specific approaches and a few computational tools. Specifically, we will focus on neural networks largely implemented in `keras`.\n",
    "\n",
    "---\n",
    "Introduction to neural networks\n",
    "---\n",
    "\n",
    "Neural networks use a series of nonlinear transformations with adjustable (trainable) parameters to approximate an input field into a desired output.\n",
    "\n",
    "Each neuron or unit of a network has an associated weight $w$ and bias $b$, and an activation function $f(z)$ for applying a nonlinear transformation such that the output is\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " f(\\mathbf{w}\\cdot \\mathbf{x} + b)\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "for input $x$.\n",
    "\n",
    "A neural network contains many layers of nodes to accomplish more involved transformations. Note that each unit only has one adjustable bias $b$, but each precedening connected note has a weight $w$ associated with it. All the weighted inputs are summed such that the output is\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " f(b+\\sum_{i} w_ix_i)\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "\n",
    "![Neural network generic example fromw ww.astroml.org](https://www.astroml.org/_images/fig_neural_network_1.png)\n",
    "<sub>Schematic from [AstroML](https://www.astroml.org/book_figures/chapter9/fig_neural_network.html#book-fig-chapter9-fig-neural-network)\n",
    "    \n",
    "    Figure produced by code under BSD license authored by Jake VanderPlas & Brigitta Sipocz.\n",
    "    The figure produced by this code is published in the textbook \"Statistics, Data Mining, and Machine Learning in Astronomy\" (2019)\n",
    "    For more information, see http://astroML.github.com"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Some examples of activation functions are \n",
    "Sigmoid:\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " f(z) = \\frac{1}{1+e^{-z}}\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEICAYAAABPgw/pAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXTdZbn28e+dOW2Spm3SeYaWtpSWlg5MCopwylwVZSgiBeUFxaOICviinCOiDEePA2DfymESGQWxQqWgMihDJ+jcpg3pkHTK1Mxzcr9/JHBiSOluu5PfHq7PWlnJ3vtp9rWh61pPn9/wmLsjIiLRLyHoACIiEh4qdBGRGKFCFxGJESp0EZEYoUIXEYkRKnQRkRihQpeYY2bzzezlSHtfM3vNzL7Sm5kkvqjQJWqZ2alm9paZVZpZuZm9aWaz3P337n5Wb+cJ6n1FPpAUdACRw2FmWcALwHXA00AK8AmgMchcIkHSDF2i1QQAd3/C3Vvdvd7dX3b3tWZ2pZn984OBZnaWmeV1zOTvN7PXP1j66Bj7ppn9t5lVmFmBmZ3c8XyhmRWb2Zc7/a5+ZvaomZWY2Q4zu9XMEjr9rs7ve6aZbe5433sB67X/OhKXVOgSrbYArWb2iJmdbWb9uxtkZjnAH4BbgIFAHnByl2FzgLUdrz8OPAnMAo4GLgfuNbOMjrG/BvoB44DTgCuABQd432eBW4Ec4H3glMP9sCKhUKFLVHL3KuBUwIHfAiVmttjMBncZeg6wwd2fc/cW4FfA3i5jtrn7Q+7eCjwFjAR+5O6N7v4y0AQcbWaJwMXALe5e7e7bgZ8BX+om4jnARnf/g7s3A7/o5n1FwkqFLlHL3Te5+5XuPgKYAgyjvTg7GwYUdvozDhR1GbOv08/1HeO6PpdB+0w7BdjR6bUdwPBu4nX3voXdjBMJGxW6xAR33ww8THuxd7YHGPHBAzOzzo8PUSnQDIzu9NwoYFc3Y/fQPtPv/L4juxknEjYqdIlKZjbRzG40sxEdj0cClwLvdBn6InCcmc0zsyTg68CQw3nPjiWZp4E7zCzTzEYD3wYe62b4i8CxZva5jvf998N9X5FQqdAlWlXTfjBzmZnV0l7k64EbOw9y91LgC8DdQBkwGVjJ4Z/e+A2gFigA/kn7QdQHuw7q9L53drzveODNw3xPkZCYNriQeNJximERMN/dXw06j0g4aYYuMc/M/s3Mss0sFfg+7eeDd12aEYl6KnSJByfRfh54KXA+MM/d64ONJBJ+WnIREYkRmqGLiMSIwG7OlZOT42PGjAnq7UVEotKqVatK3T23u9cCK/QxY8awcuXKoN5eRCQqmdmOA72mJRcRkRihQhcRiREqdBGRGKFCFxGJEQctdDN7sGPXlvUHeN3M7Fdmlm9ma81sRvhjiojIwYQyQ38YmPsxr59N+42HxgPXAL858lgiInKoDlro7v4GUP4xQy4EHvV27wDZZjY0XAFFRCQ04TgPfTj/uhNLUcdze7oONLNraJ/FM2rUqDC8tYhIZHB3GlvaqGpopqahhZrGlg+/1zW1UtvUQl1j+/cTRvfnE+O7vTboiISj0LvbybzbG8S4+yJgEcDMmTN1ExkRiUhtbc7+uiZKa5ooq22krKaJ/XVNlNc2UVHXzP669u+V9e1fVfXNVDe00NTaFtLvv+70oyK20Iv41621RgC7w/B7RUTCrqG5lV0V9eyuqGdPZQN7KhrYW9VAcVUD+6obKK5qpKy2ida27uecWWlJZPdJoX+fZLLSkxnRP51+6e0/Z6YlkZmWTFZaEn1TkshISyIjNYm+qUn0TUmkT2oS6cmJJCZ0Nw8+cuEo9MXA9Wb2JO07yFS6+0eWW0REektlfTPbSmspKKlhe1kdO8tq2VFeR2F5PaU1H92samDfFAZnpTE4K5XJQ7PIzUwlNyOVgRmpDMxIIScjlQF9U8hOTyYpMXLP9j5ooZvZE8DpQI6ZFQG3AckA7r4QWAKcA+QDdcCCngorItJZVUMzeXur2bynii37athaXE1+cQ2lNU0fjkkwGNovndED+3DGxEEM75/OiP7pDMtOZ1i/dAZlpZKWnBjgpwifgxa6u196kNed9o13RUR6TGV9M2uLKlhbVMn6XZWs21VJ0f7/3ackMy2J8YMy+PTEQRw9KIOxORmMzenLqAF9SEmK3Fl1OAV2t0URkQNxd3aU1bF8WzkrtpfzXmEF+cU1H74+emAfpo3M5rI5o5g0JIuJQzMZkpWGWc+sTUcLFbqIRITdFfX8M7+Ut/JLeev9Moqr29e6+/dJZsao/sw7fhjHj+zPcSP60S89OeC0kUmFLiKBaG5tY8X2cl7LK+HVzcVs7ZiB52SkcNJROZw4bgCzxwzg6EEZcT/zDpUKXUR6TUNzK6/llfDyhr38bXMxlfXNJCcac8YO5OJZIzl1fA7HDM5UgR8mFbqI9Kjm1jb+ubWUxWt28/KGvdQ2tdIvPZkzJg3irMlDOHV8DhmpqqJw0H9FEekReXureWZlIc+v3kVpTRNZaUmcN3UY508bxpxxA0iO4PO5o5UKXUTCpqG5lRfX7uF37+xgdWEFSQnGGZMGcdEJIzltQm7cnD4YFBW6iByxvZUNPPL2dp5cvpP9dc2My+3LredO4rPThzMwIzXoeHFDhS4ih23z3ioWvVHAn9fsprXNOWvyEK44aTQnHTVQBzYDoEIXkUO2flclv/77VpZu2EeflETmzxnN1aeOZeSAPkFHi2sqdBEJWd7eau5ZmsdfN+0jMy2Jb54xngWnjCG7T0rQ0QQVuoiEoGh/HT9/ZQt/fG8XGSlJfPvMCVx5yhiy0nTFZiRRoYvIAdU1tXD/q++z6B8FAHz1E+O47rSj6N9XM/JIpEIXkY9wdxav2c1Pl2xmb1UD844fxnfnTmR4dnrQ0eRjqNBF5F/sKKvl1ufX84+tpUwd0Y/75k/nhNEDgo4lIVChiwgALa1tLPpHAb/861ZSEhO4fd4U5s8eRUIPbZcm4adCFxHyi6u58ek1rCmq5OwpQ/iPC45lcFZa0LHkEKnQReJYW5vz4JvbuHtpHn1TErl//gzOOW5o0LHkMKnQReJUcXUDNz69hn9sLeUzkwbz088dR26mLtOPZip0kTj0xpYSvv30aqobWvjJZ4/j0tkjdal+DFChi8SRtjbnl3/byi//tpUJgzP4/VdO5JghmUHHkjBRoYvEicr6Zm54ajV/31zM52eM4MfzppCekhh0LAkjFbpIHNi6r5qvPLqSXfvruf3CY7n8xNFaYolBKnSRGPePrSV87bF3SU1O5MlrTmTmGF0kFKtU6CIx7LF3dnDb4g2MH5TB/1w5S5fuxzgVukgMcnfueimPha+/z+nH5PLrS6eTqTsjxjwVukiMaWlt4+bn1vGHVUVcNmcUP7rgWJK0IXNcUKGLxJD6plauf/xd/ra5mG+eMZ5vfWa8Dn7GERW6SIyobWzhqodXsHx7ObfPm8KXThwddCTpZSp0kRhQ1dDMgodWsLqwgl9cfDwXHj886EgSABW6SJSrrGvmioeWs2FXJfdeOp2zdXOtuBXSkRIzm2tmeWaWb2Y3d/N6PzP7s5mtMbMNZrYg/FFFpKuqhmaueHAZm3ZX8ZvLT1CZx7mDFrqZJQL3AWcDk4FLzWxyl2FfBza6+zTgdOBnZqZNB0V6UG1jCwseWsGG3VXcP38GZ04eHHQkCVgoM/TZQL67F7h7E/AkcGGXMQ5kWvvh9AygHGgJa1IR+VBDcytfeWQl7+3cz68unc5nVOZCaIU+HCjs9Lio47nO7gUmAbuBdcA33b2t6y8ys2vMbKWZrSwpKTnMyCLxrbm1jeseW8U728r4+ReP14YU8qFQCr27k1i9y+N/A1YDw4DjgXvNLOsjf8h9kbvPdPeZubm5hxxWJN61tTk3/WEtr+aVcMe845g3XWezyP8KpdCLgJGdHo+gfSbe2QLgOW+XD2wDJoYnooh84M6XNvPce7u48cwJXDZnVNBxJMKEUugrgPFmNrbjQOclwOIuY3YCZwCY2WDgGKAgnEFF4t1v3yhg0RsFfPmk0Vz/6aODjiMR6KDnobt7i5ldDywFEoEH3X2DmV3b8fpC4HbgYTNbR/sSzU3uXtqDuUXiypJ1e7hjySbOPW4ot51/rC7nl26FdGGRuy8BlnR5bmGnn3cDZ4U3mogAvLtzPzc8tZoTRvfnZ1+cRkKCyly6p1uwiUSwwvI6vvrISgZnpbHoSyeQlqwt4+TAVOgiEaq6oZmrHl5BS5vz0IJZDMxIDTqSRDjdy0UkArW1OTc8tZqC0lp+d9VsjsrNCDqSRAHN0EUi0M9f2cJfNxXzg3MncfLROUHHkSihQheJMC+s3c29r+Zz8cyRfPnkMUHHkSiiQheJIHl7q/nuM2s5YXR/fjRPpyfKoVGhi0SI6oZmrntsFX1Tk/jN/BmkJumMFjk0OigqEgHcne8+s5Yd5XU8/pU5DMpKCzqSRCHN0EUiwG//UcBLG/Zy89yJzBk3MOg4EqVU6CIBW7WjnLteyuPsKUP4yifGBh1HopgKXSRA+2ub+Mbj7zE8O527Lpqqg6ByRLSGLhIQd+e7f1hDSU0jz153MllpyUFHkiinGbpIQB58czt/3VTMLWdPYuqI7KDjSAxQoYsEYP2uSu78yyY+M2kwC04ZE3QciREqdJFeVt/Uyr8/+R4D+qZwj9bNJYy0hi7Sy3784kYKSmr5/Vfm0L9vStBxJIZohi7Si17esJffL9vJNZ8cxym66ZaEmQpdpJcUVzdw07NrOXZYFjeeNSHoOBKDVOgivcDdueXZddQ1tfLLS47XfVqkR6jQRXrB0ysL+dvmYm6aO5GjB2UGHUdilApdpIcVltfxoz9v5KRxA7lS9zeXHqRCF+lBbW3Ojc+sIcGM//riNBISdIqi9BwVukgPeuit7SzfVs4Pz5/M8Oz0oONIjFOhi/SQbaW13LN0M2dMHMRFJ4wIOo7EARW6SA9oa3O+94c1pCQm8JPPHaerQaVXqNBFesAjb29nxfb9/PD8Yxms3Yekl6jQRcJsR1ktd720mU8dk8vnZwwPOo7EERW6SBi5Ozc/u47kBC21SO9ToYuE0VMrCnm7oIxbzpnE0H46q0V6lwpdJEz2VTVwx5JNnDhuAJfMGhl0HIlDKnSRMHB3bn1+PU0tbdz5uam6gEgCEVKhm9lcM8szs3wzu/kAY043s9VmtsHMXg9vTJHI9pf1e3ll4z5uPGsCY3L6Bh1H4tRBN7gws0TgPuBMoAhYYWaL3X1jpzHZwP3AXHffaWaDeiqwSKSprG/mtsUbmDI8i6tOGRt0HIljoczQZwP57l7g7k3Ak8CFXcZcBjzn7jsB3L04vDFFItddL22mrKaROz83laRErWJKcEL52zccKOz0uKjjuc4mAP3N7DUzW2VmV3T3i8zsGjNbaWYrS0pKDi+xSARZsb2cx5ft5OpTxzJleL+g40icC6XQuzu6410eJwEnAOcC/wb8wMw+siWLuy9y95nuPjM3N/eQw4pEksaWVm55bh3Ds9O54UztQCTBC2WT6CKg8zlYI4Dd3YwpdfdaoNbM3gCmAVvCklIkAv2/1wvIL67hoQWz6JOi/dYleKHM0FcA481srJmlAJcAi7uM+RPwCTNLMrM+wBxgU3ijikSObaW13PtqPudNHcqnjtE5ABIZDjqtcPcWM7seWAokAg+6+wYzu7bj9YXuvsnMXgLWAm3AA+6+vieDiwSl/ZzzdaQmJvDD8yYHHUfkQyH9O9HdlwBLujy3sMvje4B7whdNJDL9afVu3swv4/Z5UxikOylKBNE5ViKHoKKuiR+/uJHjR2Yzf/aooOOI/AsdyRE5BHe9lMf+umYeveo4Xd4vEUczdJEQrdqxnyeW72TByWOYPCwr6DgiH6FCFwlBS2sbtz6/niFZaXxL55xLhFKhi4Tg4be2s2lPFbedP5mMVK1USmRSoYscxJ7Kev77lS2cfkwuc6cMCTqOyAGp0EUO4vYXNtLS5vzoginaUk4imgpd5GO8llfMknV7uf5TRzNqYJ+g44h8LBW6yAE0NLdy2+INjM3pyzWnjQs6jshB6eiOyAEsfP19dpTV8djVc0hNSgw6jshBaYYu0o3tpbXc/9r7nD9tGKeOzwk6jkhIVOgiXbg7ty3eQEpiAreeOynoOCIhU6GLdLF0w15e31LCDWdOYLBuviVRRIUu0kltYwv/+eeNTBySyZdPGh10HJFDooOiIp386u9b2VPZwL2XTdeGzxJ19DdWpMOWfdX8zz+28cWZIzhh9ICg44gcMhW6CO0HQn/w/Hr6piZx09yJQccROSwqdBHg+dW7WLatnJvmTmRgRmrQcUQOiwpd4l5lXTN3vLiJaSOzuWTWyKDjiBw2FbrEvf96OY/y2ibumDdFuxBJVFOhS1xbW1TBY8t2cMVJY5gyvF/QcUSOiApd4lZrm3Pr8+sZ2DeVb5+lXYgk+qnQJW49vmwHa4sq+cF5k8hKSw46jsgRU6FLXCqubuDupXmccvRALpg2LOg4ImGhQpe49JMXN9HY3MbtF2oXIokdKnSJO2/ll/L86t1ce/pRjMvNCDqOSNio0CWuNLa0cuuf1jN6YB++dvpRQccRCSvdnEviysLXCigoqeXRq2aTlqxdiCS2aIYucaOgpIb7Xsvn/GnD+OSE3KDjiISdCl3ignv7OeepSQn84DztQiSxSYUuceH51bt46/0yvjd3IoMytQuRxKaQCt3M5ppZnpnlm9nNHzNulpm1mtlF4YsocmT21zbx4xc2cfzIbObPHhV0HJEec9CDomaWCNwHnAkUASvMbLG7b+xm3F3A0p4IKnK4frJkExX1zfzus8fp5lsS00KZoc8G8t29wN2bgCeBC7sZ9w3gWaA4jPlEjshb+aU8s6qIaz45jsnDsoKOI9KjQin04UBhp8dFHc99yMyGA58FFn7cLzKza8xspZmtLCkpOdSsIoekobmV7/9xHaMH9uGbZ4wPOo5Ijwul0Lv7N6p3efwL4CZ3b/24X+Tui9x9prvPzM3VaWPSs+79ez7by+q4Y95xOudc4kIoFxYVAZ23cRkB7O4yZibwZMc9MXKAc8ysxd2fD0tKkUO0aU8VC19/n89NH86p43OCjiPSK0Ip9BXAeDMbC+wCLgEu6zzA3cd+8LOZPQy8oDKXoLS0tnHTs2vpl57MD86bHHQckV5z0EJ39xYzu572s1cSgQfdfYOZXdvx+seum4v0tofe3M7aokp+fel0+vdNCTqOSK8J6V4u7r4EWNLluW6L3N2vPPJYIodne2ktP3slj89MGsx5U4cGHUekV+lKUYkZ7s4tz60jOSGBH8/Tfc4l/qjQJWb8ftlO3i4o45ZzJjGkny7vl/ijQpeYUFhex0+XbOLUo3O4dPbIg/8BkRikQpeo5+7c/NxaAO78/HFaapG4pUKXqPfE8kLezC/j++dOYkT/PkHHEQmMCl2iWmF5HT9ZsomTjxrIZbqTosQ5FbpErbY25zvPrAHgrs9P1VKLxD0VukStB9/cxrJt5fzw/MmMHKClFhEVukSlrfuquXtp+wVEXzhhRNBxRCKCCl2iTnNrGzc8vZqM1CR++jmd1SLygZAu/ReJJL/46xbW76pi4eUzyM1MDTqOSMTQDF2iyrKCMu5/7X2+OHMEc6foXi0inanQJWpU1jVzw1OrGT2gD7edf2zQcUQijpZcJCq4O99/fh3F1Y08e93J9E3VX12RrjRDl6jwzKoiXly7hxvOnMC0kdlBxxGJSCp0iXhb91Xzwz+t56RxA7n2tKOCjiMSsVToEtHqm1q5/vH36JuSxC8vOZ7EBJ2iKHIgWoiUiPajFzaQt6+aR66azaAs3eNc5ONohi4R6/n3dvHE8kKuO/0oTpuQG3QckYinQpeIlLe3mlueW8esMf359pkTgo4jEhVU6BJxqhqaufaxVWSkJXHfZTNITtRfU5FQaA1dIoq7891n1rCzvI4nvnqi1s1FDoGmPhJRfvP6+yzdsI9bzp7I7LEDgo4jElVU6BIx/rZpH/cszeO8qUO5+tSxQccRiToqdIkI+cXVfPPJ1UwemsU9F03TLXFFDoMKXQJXWdfMVx9dRVpyAouumEl6SmLQkUSikg6KSqCaW9v42uOrKNpfx+NfPZHh2elBRxKJWip0CYy7c+sf1/Nmfhn3XDSVWWN0EFTkSGjJRQLzm9ff56mVhXzj00fzhZkjg44jEvVU6BKIF9bu5u6X8rhg2jBdCSoSJip06XVv5pdyw1OrmT1mAHdfNFVntIiESUiFbmZzzSzPzPLN7OZuXp9vZms7vt4ys2nhjyqxYF1RJdc8upJxORn89oqZpCXrjBaRcDlooZtZInAfcDYwGbjUzCZ3GbYNOM3dpwK3A4vCHVSi37bSWq58aDnZfVJ49OrZ9OuTHHQkkZgSygx9NpDv7gXu3gQ8CVzYeYC7v+Xu+zsevgOMCG9MiXZF++u4/IFltLnz6NWzGax7tIiEXSiFPhwo7PS4qOO5A7ka+Et3L5jZNWa20sxWlpSUhJ5SotreygbmP7CMqoZmfnf1HI7KzQg6kkhMCqXQuzti5d0ONPsU7YV+U3evu/sid5/p7jNzc7VhQTworWlk/gPvUFrdyCNXzWbK8H5BRxKJWaFcWFQEdD5JeASwu+sgM5sKPACc7e5l4Ykn0aykur3Md1XU88iC2cwY1T/oSCIxLZQZ+gpgvJmNNbMU4BJgcecBZjYKeA74krtvCX9MiTZ7Kxu4eNHbFJbX8+CXZzFn3MCgI4nEvIPO0N29xcyuB5YCicCD7r7BzK7teH0h8ENgIHB/xznFLe4+s+diSyTbVVHPZb/932UW3ddcpHeYe7fL4T1u5syZvnLlykDeW3pOfnENX35wOVUNzTxylZZZRMLNzFYdaMKsm3NJ2KwurGDBQ8tJTDCe+OqJOgAq0stU6BIWb2wp4drHVjEwI4XfXTWHMTl9g44kEndU6HLEnli+k1ufX8+EwZk8smCWNnYWCYgKXQ5ba5tz10ubWfRGAadNyOXey6aTmabL+UWCokKXw1Ld0My3n17DKxv3ccVJo/nheZNJStTNO0WCpEKXQ5ZfXMP/+d1KtpfV8R/nT+bKU8YGHUlEUKHLIXpp/V6+88waUpMSeOzqOZx0lC4YEokUKnQJSWNLK3f+ZTMPvbmdaSOzWXj5DIb204bOIpFEhS4HVVBSwzeeeI8Nu6tYcMoYbj57IqlJ2phCJNKo0OWA3J0nlhfy4xc3kpKUwG+vmMmZkwcHHUtEDkCFLt3aW9nATc+u5fUtJZxy9EDuuWgaw7K1xCISyVTo8i/a2pynVxbykyWbaG51fnThsVw+ZzQJCdrIWSTSqdDlQ/nF1Xz/ufUs317O7LEDuOvzUxmrS/hFooYKXahpbOHev+fz4D+3kZ6SyN2fn8oXZo6g41bIIhIlVOhxrK3Nee69Xdz10mZKqhv5/IwR3HLORHIyUoOOJiKHQYUeh9yd17aUcPdLeWzaU8W0kdks+tIJTNe9y0Wimgo9zqzYXs7PXs7jnYJyRg5I5xcXH88F04bpoKdIDFChx4l3Csr45V+38nZBGTkZKfzH+ZO5bM5oUpJ0Qy2RWKFCj2Gtbc4rG/ey6I0C3t1ZQW5mKreeO4n5c0aTnqIrPUVijQo9BlXWN/PsqiIeeXs7O8rqGDWgD/95wbFcPGskackqcpFYpUKPEe7O2qJKnli+k+dX76KhuY0Zo7K5ee5Ezjp2CIlaIxeJeSr0KFdS3cifVu/i6ZWFbNlXQ3pyIp+dPpz5c0Zrk2aROKNCj0KVdc0s3bCXxWt289b7pbQ5HD8ymzs+O4Xzpg6jX7q2gROJRyr0KLGvqoGXN+7j5Q17efv9MlranFED+vC104/mwuOHMX5wZtARRSRgKvQI1dzaxprCCl7LK+HVvGI27K4CYGxOX67+xFjOnjKUaSP66fJ8EfmQCj1CtLS2sWlPNcu2lfHW+2UsKyijtqmVxATjhFH9+d7cYzhj4mAmDM5QiYtIt1ToAamoa2J1YQXv7azg3Z37eW9nBTWNLUD7LPyzM4ZzylE5nHxUDv36aE1cRA5Ohd7D3J2S6kY27a1m4+4q1u+qZN2uSnaW1wGQYDBhcCbzpg9j9tiBzB4zgCH90gJOLSLRSIUeJm1tzp6qBraV1FJQWsPWfTVs2VfN1uIaymubPhw3on86xw3vx8WzRjJ9VDZTR2STkar/DSJy5NQkIXJ3qupb2FVRz66Keor211FYXs/O8jp2lteys7yOhua2D8dnpiUxYXAmZ00ezMQhmUwcmsXEIZlk90kJ8FOISCyL+0Jva3Mq6pspq2mktKaJkppGSqsbKa5upLiqgX3VDeypbGBPRQP1za3/8mfTkhMYPaAvowf25ZPjcxmXm8HYnL6My+3LoMxUHbwUkV4VUqGb2Vzgl0Ai8IC739nldet4/RygDrjS3d8Nc9ZuuTuNLW3UNrZQ29hKdWMzNQ0tVDe0UN3YTFV9C1X1zVTWN1NR30xFXTMVdU3sr2uioq6Z/XVNtPlHf29yojEoM43BWakcMziT0ycMYlh2GkP7pTOifzrD+6czsG+KSltEIsZBC93MEoH7gDOBImCFmS12942dhp0NjO/4mgP8puN72L2aV8yPX9hIXVNrx1cLza3dNHIXfVISyU5PJis9mf59UjimY/ljYN8UBnR85WSkMigzlZyMVLL7JKusRSSqhDJDnw3ku3sBgJk9CVwIdC70C4FH3d2Bd8ws28yGuvuecAful57MxKFZ9E1JpE9KEn1SEumbmkRGatKH3zPT2r9npSeTlZZEZlqy7vstIjEvlEIfDhR2elzER2ff3Y0ZDvxLoZvZNcA1AKNGjTrUrADMGNWfGZdpqzQRka5CmbZ2t+7QdY0jlDG4+yJ3n+nuM3Nzc0PJJyIiIQql0IuAkZ0ejwB2H8YYERHpQaEU+gpgvJmNNbMU4BJgcZcxi4ErrN2JQGVPrJ+LiMiBHXQN3d1bzOx6YCntpy0+6O4bzOzajtcXAktoP2Uxn/bTFhf0XGQREelOSOehu/sS2ku783MLO/3swNfDG01ERA6FzuUTEYkRKnQRkRihQhcRicyG/WgAAAKzSURBVBHWvvwdwBublQA7AnnzI5MDlAYdopfpM8e+ePu8EL2febS7d3shT2CFHq3MbKW7zww6R2/SZ4598fZ5ITY/s5ZcRERihApdRCRGqNAP3aKgAwRAnzn2xdvnhRj8zFpDFxGJEZqhi4jECBW6iEiMUKEfATP7jpm5meUEnaUnmdk9ZrbZzNaa2R/NLDvoTD3FzOaaWZ6Z5ZvZzUHn6WlmNtLMXjWzTWa2wcy+GXSm3mJmiWb2npm9EHSWcFGhHyYzG0n7Pqs7g87SC14Bprj7VGALcEvAeXpEp/1zzwYmA5ea2eRgU/W4FuBGd58EnAh8PQ4+8we+CWwKOkQ4qdAP338D36ObnZlijbu/7O4tHQ/foX0Dk1j04f657t4EfLB/bsxy9z3u/m7Hz9W0F9zwYFP1PDMbAZwLPBB0lnBSoR8GM7sA2OXua4LOEoCrgL8EHaKHHGhv3LhgZmOA6cCyYJP0il/QPiFrCzpIOIV0P/R4ZGZ/BYZ089L/Bb4PnNW7iXrWx31ed/9Tx5j/S/s/0X/fm9l6UUh748YiM8sAngW+5e5VQefpSWZ2HlDs7qvM7PSg84STCv0A3P0z3T1vZscBY4E1Zgbtyw/vmtlsd9/bixHD6kCf9wNm9mXgPOAMj92LF+Jyb1wzS6a9zH/v7s8FnacXnAJcYGbnAGlAlpk95u6XB5zriOnCoiNkZtuBme4ejXdtC4mZzQV+Dpzm7iVB5+kpZpZE+0HfM4BdtO+ne5m7bwg0WA+y9lnJI0C5u38r6Dy9rWOG/h13Py/oLOGgNXQJxb1AJvCKma02s4UH+wPRqOPA7wf7524Cno7lMu9wCvAl4NMd/29Xd8xcJQpphi4iEiM0QxcRiREqdBGRGKFCFxGJESp0EZEYoUIXEYkRKnQRkRihQhcRiRH/HxkRClUldXc+AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-5,5,100)\n",
    "y = 1/(1+np.exp(-x))\n",
    "plt.plot(x,y)\n",
    "plt.title('Sigmoid')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or\n",
    "\n",
    "Rectified Linear Unit (ReLU):\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " f(z) = \\max(0,z)\n",
    "\\end{aligned}\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWoAAAEICAYAAAB25L6yAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAXWElEQVR4nO3de5zWY/7H8ddHSouspXEqlV2HFXYdhiX2t1ayUmv34ZBSIoda51kROfWInHIoFikhSdJx2UTlFKF2R3JI1mkdtqQhUUKqz++Pa2Ydmpp7Zu7vfX3v+34/H495NHPP3X2/b9XHNZ/7+n4uc3dERCS9NogdQERE1k+FWkQk5VSoRURSToVaRCTlVKhFRFJOhVpEJOVUqEVEUk6FWvKSmb1nZl+Z2XIzW2RmI8xs0wx+39Nmduo6Hu/QH912kpnNzGZukbpQoZZ89kd33xTYE9gL6Bs5j0giVKgl77n7ImAqoWBjZvub2fNmttTMXjazg6MGFKknFWrJe2bWHGgPvG1mzYBHgAHAFsD5wAQzK4kYUaReVKgln/3dzJYBHwKLgX5AN2CKu09x9zXuPh0oB46ImFOkXlSoJZ/92d2bAAcDvwSaAi2BYyvbHkvNbClwELBtDY+1Cmj4o9saAt9mN7JI7W0YO4BIfbn7DDMbAdwAzAbuc/fTavkwHwCtfnTbDsD79Q4oUk9aUUuhGAy0A2YCfzSzP5hZAzNrbGYHV/axq2xYeXvVR0PgQaDMzH5pQSlwMjAm9y9F5IdUqKUguHsFMBIoA/4EXAxUEPrXF/DDv+tDgK++93EPcGflr/8APq98rEvc/bEcvQSRdTIdHCAikm5aUYuIpJwKtYhIyqlQi4iknAq1iEjKJbKPumnTpt6qVaskHlpEpCC9+OKLn7h7taMOEinUrVq1ory8PImHFhEpSGa2zour1PoQEUk5FWoRkZRToRYRSTkVahGRlFOhFhFJuYx2fZjZe8AyYDWwyt1LkwwlIiLfqc32vN+7+yeJJRERkWqp9SEikgXPPguDBkESA0kzLdQOTDOzF82sZ3V3MLOeZlZuZuUVFRXZSygiknKLFkGnTjBkCKxYkf3Hz7T1caC7LzSzrYDpZvaGuz/z/Tu4+zBgGEBpaamGXItIUVi1Cjp3hs8/h2nTYJNNsv8cGa2o3X1h5a+LgUnAftmPIiKSfy67DGbMgKFDYY89knmOGgu1mW1iZk2qPgcOA15LJo6ISP54+GG49lro1QtOOCG558mk9bE1MMnMqu4/WufIiUixe+cd6N4d9tkHBg9O9rlqLNTu/i7w62RjiIjkj6++gmOOgQ02gHHjoHHjZJ8vkTGnIiKF7OyzYe5cmDwZdtgh+efTPmoRkVq45x646y645BLo0CE3z6lCLSKSoblz4YwzoG1b6N8/d8+rQi0ikoGlS0NfesstYfRoaNAgd8+tHrWISA3c4aST4P33w57prbbK7fOrUIuI1OD66+Ghh8I2vDZtcv/8an2IiKzHjBnQty8ceyycc06cDCrUIiLr8NFHcNxxsNNOYadHuO4v99T6EBGpRtWwpWXL4IknoEmTeFlUqEVEqnHxxfDMMzBqFOy2W9wsan2IiPzI3/8e3kD8y1+ga9fYaVSoRUR+4O234cQTobQ0+WFLmVKhFhGpVDVsacMNYfx42Gij2IkC9ahFRCqdeSa88go88gi0bBk7zXe0ohYRIWy/u+ceuPRSaN8+dpofUqEWkaL30kthNd2uHfTrFzvN2lSoRaSoVQ1bKimB++/P7bClTKlHLSJFa82asMPjgw/CnumSktiJqqdCLSJFa+DAcEDtzTfDAQfETrNuan2ISFF66qlwSkunTuForTRToRaRorNwYZjjsfPOMHx4vGFLmVLrQ0SKyrffhol4y5fDk0/GHbaUKRVqESkqffvCzJlhh0fsYUuZUutDRIrGxIlw443hgNrjj4+dJnMq1CJSFN56C3r0gP32g5tuip2mdlSoRaTgrVgRLmpp2BDGjUvPsKVMqUctIgXNPbQ6Xn0VpkyBFi1iJ6o9rahFpKANHw733guXXw6HHx47Td2oUItIwZozJ1zMcthhcNllsdPUnQq1iBSkzz6Do4+GrbZK77ClTKlHLSIFZ80a6N4dFiyAZ5+Fpk1jJ6qfjFfUZtbAzF4ys8lJBhIRqa9rr4XJk8M2vN/8Jnaa+qtN6+NcYH5SQUREsuGJJ0I/unPncBhAIcioUJtZc6ADMDzZOCIidbdgAXTpArvsAnfemf5hS5nKdEU9GOgDrFnXHcysp5mVm1l5RUVFVsKJiGSqatjSihXhBPFNN42dKHtqLNRm1hFY7O4vru9+7j7M3UvdvbQkrcckiEjBuvBCeO65sG+6devYabIrkxX1gcCRZvYeMAY4xMxGJZpKRKQWxo+HQYPgrLNCb7rQ1Fio3b2vuzd391ZAZ+BJd++WeDIRkQy8+SacfHLY3XHjjbHTJEMXvIhI3vryy3BRS6NGYdhSo0axEyWjVhe8uPvTwNOJJBERqQV3OP10mDcPpk6F7bePnSg5ujJRRPLSsGFw333Qvz+0axc7TbLU+hCRvFNeDuecE6bhXXpp7DTJU6EWkbyyZEk4BGCbbWDUKNigCKqYWh8ikjfWrIFu3WDhwnBA7ZZbxk6UGyrUIpI3rr4aHn0UbrstnH1YLIrghwYRKQSPPx5OaenaNez2KCYq1CKSeh9+GIYt7borDB1aOMOWMqVCLSKptnJlGLb09dcwcSJssknsRLmnHrWIpFqfPvDCCzB2bBhfWoy0ohaR1Bo7Fm6+Gc49F449NnaaeFSoRSSV3ngDTjkFDjgABg6MnSYuFWoRSZ3ly8OwpcaNw6q6UIctZUo9ahFJFXfo1Qvmz4dp06B589iJ4lOhFpFUGTIERo+GK6+EQw+NnSYd1PoQkdT45z+hrAyOOAIuvjh2mvRQoRaRVPj007CzY7vtwvjSYhi2lCm1PkQkutWrw6XhixaFA2q32CJ2onRRoRaR6AYMCKe0DBkCpaWx06SPfrgQkaimTg2ntHTrFnZ7yNpUqEUkmg8+CC2P3XaDO+4ovmFLmVKhFpEoVq6ETp3Cr+PHF+ewpUypRy0iUfTuDbNnw7hxxTtsKVNaUYtIzo0ZA7feGvZMH3NM7DTpp0ItIjn1+utw6qnQpo2GLWVKhVpEcmb58rCC3njjMGypYcPYifKDetQikhPucNpp8O9/w/Tp0KxZ7ET5Q4VaRHLitttCb/qqq+CQQ2KnyS9qfYhI4mbNgvPOg44d4aKLYqfJPyrUIpKoTz4J+6WbNYORIzVsqS7U+hCRxFQNW1q8GJ5/Hn72s9iJ8lONhdrMGgPPABtV3n+8u/dLOpiI5L8rrwyntAwbBnvvHTtN/spkRf0NcIi7LzezhsBMM3vU3WclnE1E8thjj8EVV8CJJ4Z901J3NRZqd3dgeeWXDSs/PMlQIpLfqoYt7bEH3H67hi3VV0ZtfTNrYGZzgcXAdHefXc19eppZuZmVV1RUZDuniOSJb74JJ7WsWhWGLW28cexE+S+jQu3uq919T6A5sJ+Z7V7NfYa5e6m7l5aUlGQ7p4jkid69w9mH99wDO+0UO01hqNVGGXdfCjwNHJ5IGhHJa6NHhwtbeveGo46KnaZw1FiozazEzDav/PwnwKHAG0kHE5H8Mm9euET8oIPgmmtipyksmez62Ba418waEAr7WHefnGwsEckny5bB0UdDkybw4IMatpRtmez6eAXYKwdZRCQPuYftd2+9BU88AdttFztR4dGViSJSL3/7WxhZes01cPDBsdMUJl11LyJ19sIL4Y3DI4+EPn1ipylcKtQiUicVFWHYUosWcO+9GraUJLU+RKTWVq+G448PxfqFF2DzzWMnKmwq1CJSa/37w+OPw/DhsJe2GiROP6yISK1MmRKm4vXoAaecEjtNcVChFpGMvfcedOsGv/51uAJRckOFWkQyUjVsafXqMGzpJz+Jnah4qEctIhkpK4Pycpg0CXbcMXaa4qIVtYjUaNQouOMOuOAC+POfY6cpPirUIrJer70GvXrB//0fXH117DTFSYVaRNbpiy/CsKXNNoMxY2BDNUuj0H92EamWe9h+98478OSTsO22sRMVLxVqEanWzTeH3R0DB4a2h8Sj1oeIrOX557974/D882OnERVqEfmBxYvDfumWLcO5hzpBPD61PkTkf6qGLS1ZArNmadhSWqhQi8j/9OsXTmm5++5wmbikg1ofIgLAI4/AVVeFY7V69IidRr5PhVpE+M9/wrClvfYKR2tJuqhQixS5r7+GY44Jn48fD40bx80ja1OPWqTIlZXBnDnw0EPw85/HTiPV0YpapIjddx8MHQoXXhgOqJV0UqEWKVKvvhqGLf3udzBgQOw0sj4q1CJF6PPPw7ClzTfXsKV8oD8ekSLjDiefDO++C089BdtsEzuR1ESFWqTIDBoEEyfCDTfAb38bO41kQq0PkSIycyb06QNHHQXnnRc7jWRKhVqkSHz8MXTqBDvsEC4R17Cl/KHWh0gRWLUKunSBpUvhscfgpz+NnUhqQ4VapAhcfnl443DECPjVr2KnkdqqsfVhZtub2VNmNt/M5pnZubkIJiLZ8fDDcM01cNppcOKJsdNIXWSyol4F9Hb3OWbWBHjRzKa7++sJZxORenr3XejeHfbeG265JXYaqasaV9Tu/pG7z6n8fBkwH2iWdDARqZ+qYUtmGraU72rVozazVsBewOxqvtcT6AnQokWLLEQTkfo4+2x46SX4xz/CTg/JXxlvzzOzTYEJQJm7f/Hj77v7MHcvdffSkpKSbGYUkVoaMQKGD4e+faFjx9hppL4yKtRm1pBQpO9394nJRhKR+nj5ZTj9dPj97+GKK2KnkWzIZNeHAXcB8939puQjiUhdff556EtvsQU88ICGLRWKTFbUBwInAIeY2dzKjyMSziUiteQezjp87z0YOxa23jp2IsmWGv9/6+4zAV1sKpJyN94IkybBTTfBgQfGTiPZpFkfIgXgmWfgootC26OsLHYayTYVapE8t2gRHHcc/OIXcNddGrZUiPRWg0geW7UKOncObyJOmwabbRY7kSRBhVokj116KcyYASNHwh57xE4jSVHrQyRPPfQQXHddOKD2hBNip5EkqVCL5KF33gmT8PbZBwYPjp1GkqZCLZJnvvoq7O7YYAMNWyoW6lGL5JmzzoK5c2HyZGjVKnYayQWtqEXyyN13h49LLoEOHWKnkVxRoRbJE3PnwplnQtu20L9/7DSSSyrUInlg6dLQl95ySxg9Gho0iJ1Ickk9apGUc4eTToL33w97prfaKnYiyTUVapGUu/76sGd68GBo0yZ2GolBrQ+RFJsxI5zS0qkTnHNO7DQSiwq1SEp99FEYtrTTTuFYLQ1bKl5qfYikUNWwpWXL4PHHoUmT2IkkJhVqkRS6+OIwY3rUKNh999hpJDa1PkRSZtKk8Abi6adD166x00gaqFCLpMjbb4etePvuC4MGxU4jaaFCLZISK1bA0UeHk8PHjYONNoqdSNJCPWqRFHAPl4e/+io88gi0bBk7kaSJVtQiKXDXXTBiRDixpX372GkkbVSoRSKbMyeMLm3XDvr1i51G0kiFWiSizz4LfemSErj/fg1bkuqpRy0SyZo10L07LFgQ9kyXlMROJGmlQi0SyXXXhVNabrkF9t8/dhpJM7U+RCJ46qnwxmHnzqE/LbI+KtQiObZgQSjQO+8Mw4Zp2JLUTK0PkRz69tswEe/LL8OqWsOWJBMq1CI5dNFF8Nxz4Tit1q1jp5F8UWPrw8zuNrPFZvZaLgKJFKoJE+Cmm8IViF26xE4j+SSTHvUI4PCEc4gUtDffhB49YL/94MYbY6eRfFNjoXb3Z4AlOcgiUpCqhi01aqRhS1I3WetRm1lPoCdAixYtsvWwInnNPcyVnjcPHn0U9E9D6iJr2/PcfZi7l7p7aYkusRIB4M47YeRIuPxy+MMfYqeRfKV91CIJKS+Hs8+Gww6Dyy6LnUbymQq1SAKWLIFjjoGtt9awJam/TLbnPQC8AOxiZv81s1OSjyWSv9asgRNOgIULYfx4aNo0diLJdzW+meju2vEpUgvXXANTpsCtt4bteCL1pdaHSBY98UR447BLFzjjjNhppFCoUItkyYIFoUDvsouGLUl2qVCLZMG330KnTuHilgkTYNNNYyeSQqKhTCJZ0KcPPP88jBkDu+4aO40UGq2oRepp7FgYPDjsmT7uuNhppBCpUIvUwxtvwCmnhKO0brghdhopVCrUInX05ZfhopbGjcOqulGj2ImkUKlHLVIH7tCrF7z+OkydCttvHzuRFDIVapE6uOOOcGl4//7Qrl3sNFLo1PoQqaV//QvKyqB9+3CSuEjSVKhFauHTT0Nfeptt4L77YAP9C5IcUOtDJENVw5YWLYKZM2HLLWMnkmKhQi2SoauuCqe0DBkC++4bO40UE/3gJpKB6dOhXz/o1i3s9hDJJRVqkRp8+CEcfzy0bh12e2jYkuSaCrXIeqxcGYYtffNNGLa0ySaxE0kxUo9aZD0uuABmzQpXHu6yS+w0Uqy0ohZZhzFj4JZbwp7pY4+NnUaKmQq1SDXmz4dTT4U2bWDgwNhppNipUIv8yPLl4aKWjTcOLY+GDWMnkmKnHrXI97hDz55hfOm0adCsWexEIirUIj9w++3wwAMwYAC0bRs7jUig1odIpdmz4a9/hQ4doG/f2GlEvqNCLQJ88knY2dGsGYwcqWFLki5qfUjRW706XBr+8cfhgNottoidSOSHVKil6A0YEE5pGToU9tkndhqRtekHPClqU6eGU1q6d4fTToudRqR6KtRStD78ELp2hd13D6NLNWxJ0kqFWorSypXhzcOVK2H8+HBxi0haqUctRal377Adb8IE2Hnn2GlE1k8raik6DzwAt94K550HRx0VO41IzTIq1GZ2uJn928zeNrOLkg4lkpTXXw/Dlg46CK69NnYakczUWKjNrAFwG9AeaA10MbPWSQcTSUKjRmEi3oMPatiS5I9MetT7AW+7+7sAZjYG+BPwerbDlJXB3LnZflQR2HNPGDwYdtwxnH8okk8yaX00Az783tf/rbztB8ysp5mVm1l5RUVFtvKJiBS9TFbU1e0u9bVucB8GDAMoLS1d6/uZGDy4Lr9LRKSwZbKi/i+w/fe+bg4sTCaOiIj8WCaF+l/ATma2g5k1AjoDDycbS0REqtTY+nD3VWZ2FjAVaADc7e7zEk8mIiJAhlcmuvsUYErCWUREpBq6MlFEJOVUqEVEUk6FWkQk5VSoRURSztzrdG3K+h/UrAJ4P+sPnKymwCexQ+SYXnNx0GvODy3dvaS6byRSqPORmZW7e2nsHLmk11wc9Jrzn1ofIiIpp0ItIpJyKtTfGRY7QAR6zcVBrznPqUctIpJyWlGLiKScCrWISMqpUFfDzM43MzezprGzJM3MrjezN8zsFTObZGabx86UhGI7oNnMtjezp8xsvpnNM7NzY2fKFTNrYGYvmdnk2FmyRYX6R8xse6Ad8EHsLDkyHdjd3X8FvAn0jZwn64r0gOZVQG933xXYHzizCF5zlXOB+bFDZJMK9doGAX2o5rixQuTu09x9VeWXswgn+BSa/x3Q7O4rgaoDmguWu3/k7nMqP19GKFxrnXVaaMysOdABGB47SzapUH+PmR0JLHD3l2NnieRk4NHYIRKQ0QHNhcrMWgF7AbPjJsmJwYSF1prYQbIpo4MDComZPQ5sU823LgEuBg7LbaLkre81u/tDlfe5hPDj8v25zJYjGR3QXIjMbFNgAlDm7l/EzpMkM+sILHb3F83s4Nh5sqnoCrW7H1rd7Wa2B7AD8LKZQWgBzDGz/dx9UQ4jZt26XnMVMzsR6Ai09cLcWF+UBzSbWUNCkb7f3SfGzpMDBwJHmtkRQGNgMzMb5e7dIueqN13wsg5m9h5Q6u75NoGrVszscOAm4HfuXhE7TxLMbEPCG6VtgQWEA5uPL+SzPy2sNu4Flrh7Wew8uVa5oj7f3TvGzpIN6lHLrUATYLqZzTWzO2IHyrbKN0urDmieD4wt5CJd6UDgBOCQyj/XuZUrTclDWlGLiKScVtQiIimnQi0iknIq1CIiKadCLSKScirUIiIpp0ItIpJyKtQiIin3//gX3ZzIwd5oAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(-5,5,100)\n",
    "plt.plot(x[:50],np.zeros(50),'b-')\n",
    "plt.plot(x[50:],x[50:],'b-')\n",
    "plt.title('ReLU')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And many others, which is beyond the scope of this brief tutorial. There are many excellent resources on the choice activation functions (e.g. [Neural Networks and Deep Learning by Michael Nielson](http://neuralnetworksanddeeplearning.com/chap3.html))\n",
    "\n",
    "---\n",
    "\n",
    "\n",
    "Packages like PyTorch and TensforFlow provide the tools to contruct neural networks in Python. However, here we focus on Keras, a higher-level package which makes it easier to contruct a network.\n",
    "\n",
    "Here as an example we will contruct a simple model for the quintessential machine learning example of identifying handwritten digits (MNIST dataset)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1) Get data and develop neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "#--  import required packages\n",
    "import keras\n",
    "from keras.datasets import mnist\n",
    "from keras.layers import Dense, Activation\n",
    "from keras.utils import plot_model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#-- Get built-in MNIST data from keras\n",
    "#-- \"Dataset of 60,000 28x28 grayscale images of the 10 digits, along with a test set of 10,000 images.\"\"\n",
    "#-- https://keras.io/datasets/#mnist-database-of-handwritten-digits\n",
    "(x_train, y_train), (x_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA4sAAABDCAYAAAA1QJAnAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOy9d3Rc1bm//5wZ9d671bstW3KRC+7GxrgACT0JvYTcBQkEbgjhd28SLuSbwEoIgXAJxTRDgOsA7gb3IlmuapasXqzeZ0ajKZpyfn/Ic2LhCkhzRnCetWZhxrLmo6199t7vu98iiKKIgoKCgoKCgoKCgoKCgsK5qOQWoKCgoKCgoKCgoKCgoOB6KMaigoKCgoKCgoKCgoKCwnkoxqKCgoKCgoKCgoKCgoLCeSjGooKCgoKCgoKCgoKCgsJ5KMaigoKCgoKCgoKCgoKCwnkoxqKCgoKCgoKCgoKCgoLCeSjGooKCgoKCgoKCgoKCgsJ5jLuxKAjCPkEQTIIg6M++qsf7M78tgiCECILwmSAIQ4IgNAuC8CO5NV0pgiCknR3v9XJruRyCIDwsCMJxQRDMgiC8I7eeK0EQhCxBEPYIgqAVBKFOEIQfyK3pUgiC4CkIwltn5/GgIAjFgiBcK7euyzFB58Z6QRA6BEHQCYJQIwjC/XJruhwTcZwdTLC1biLOjYm4d0+4cQYQBOE2QRBOnz1z1AuCsEBuTZdioq0b58xhx8smCMLLcuu6FBN4704UBGGbIAgDgiB0CoLwiiAIbnLruhwT8Bl06lnUWTeLD4ui6Hf2leGkz/w2/B0YBiKBHwP/KwjCZHklXTF/B47JLeIKaQeeBdbJLeRKOLvgbQS2ACHAg8B6QRDSZRV2adyAFmAREAj8F/CJIAiJMmq6EibU3DjL/wMSRVEMAK4DnhUEYYbMmi7HRBxnBxNprZuIcwMm3t494cZZEITlwJ+AewB/YCHQIKuoyzOh1o1z5rAfI+c6I/B/Msu6HBN1734V6AaigVxG9P+HrIouw0R7BuU4iyphqF9BEARf4Ebgv0RR1IuieAjYBNwhr7LLIwjCbYAG2C23litBFMVPRVH8HOiTW8sVkgnEAC+KomgTRXEPUIALzw1RFIdEUfydKIpNoijaRVHcAjQCLn2AmoBzA1EUK0RRNDv+9+wrRUZJl2UijjNMyLVuws2NicgEHeffA8+Iolh0do1uE0WxTW5Rl2KirhtnuYkRY+ag3EIuxUTdu4Ek4BNRFE2iKHYCOwBXv2yZaM+g08+izjIW/58gCL2CIBQIgrDYSZ/5TUkHbKIo1pzzXikuPtkFQQgAngEel1vLdxjhIu9NcbaQb4ogCJGMzPEKubV8FxEE4VVBEAxAFdABbJNZ0neOibrWTdC5MZH2bmBijbMgCGpgJhB+NpSs9WzYnrfc2r7D3AW8J4qiKLeQr8ME2rtfAm4TBMFHEIRY4FpGDEaXZII+g04/izrDWHwSSAZigdeBzYIguLKnzw/QfuU9LSNX067M/wBviaLYIreQ7zBVjHgk/1MQBHdBEFYwEmLhI6+sK0MQBHfgA+BdURSr5NbzXUQUxf9gZK1YAHwKmC/9LxS+ARNyrZuAc2Oi7d3AhBvnSMCdkduuBYyE7eUB/5+cor6rCIIQz8ie/a7cWr4OE2zv3s/I5YoOaAWOA5/LqujSTMRn0Oln0XE3FkVRPCKK4qAoimZRFN9l5Kp01Xh/7rdADwR85b0AYFAGLVeEIAi5wNXAi3Jr+S4jiqIFuAFYDXQycrPxCSMLoksjCIIKeJ+RXNyHZZbzneZsWMghIA74mdx6vktM9LVuIs2NCbh3S0ygcTae/e/Loih2iKLYC/yFCTLOE5A7gUOiKDbKLeRKmUh791mtXzDipPEFwoBgRvIBXZUJ9wzKcRaVo0KRyIWvUF2FGsBNEIQ0URRrz743Dde++l8MJAJnBEGAkdtRtSAI2aIoTpdR13cOURTLGPHgACAIQiEu7qUURibFW4x40FadXWgUxh83XD9faqKxmO/GWjcR54ar790XwqXHWRTFAUEQWhkZW4Xx507gj3KLuFIm4N4dAkwCXjmbO2wWBOFtRooh/UpWZRdhoj6Dzj6LjuvNoiAIQYIgXCMIgpcgCG6CIPyYkSpDX4zn534bRFEcYsQr8owgCL6CIFwFXM+IZ8dVeZ2RDTH37Os1YCtwjZyiLsfZOeEFqBk58Hm5eollQRCmntXpIwjCE4xU/HpHZlmX43+BLGCtKIrGy32xKzDR5oYgCBFnS2/7CYKgFgThGuB2YI/c2i7FRBtnJuBaNxHnxkTcuyfiOJ/lbeCRs/qDgUcZqXLoskzAdQNBEOYxElLt6lVQz2VC7d1nb+UagZ+dnSNBjOSIlsqr7LJMxGfQuWdRURTH7QWEM1LafJCRynVFwPLx/Mwx0h3CSIz1EHAG+JHcmr6m/t8B6+XWcYU6xa+8fie3rstofgEYYCRceTuQKremy+hNODuuprOaHa8fy63tuzQ3zq51+8+uczqgHHhAbl3ftXG+iH6XXusm4tyYiHv3RBzns7rdGWk3oGEkpOxvgJfcui6jecKtG8A/gPfl1vE19E7UvTsX2Hf2nNTLiHEeIbeuy2ieiM+gU8+iwtkPVVBQUFBQUFBQUFBQUFCQUPosKigoKCgoKCgoKCgoKJyHYiwqKCgoKCgoKCgoKCgonIdiLCooKCgoKCgoKCgoKCich2IsKigoKCgoKCgoKCgoKJzH5Uodu2r1m0v1elI0jx2KZuegaHYOimbnoGh2Dopm56Bodg6KZuegaHYO3ynNys2igoKCgoKCgoKCgoKCwnm4dBNVBQUFBQUFBYXvC3a7HaPRSHl5OVarFT8/P3Jzc+WWpaCg8D1GMRYVFBQUFBQUFGRGFEUMBgNNTU389Kc/RavVMmXKFLZs2SK3NAWF7x12ux1HL3q1Wi2zGnlRjEUFBQUFBQUFBZn55z//ya5du9i/fz9tbW1kZGSQmpoqtywFhe8VJpOJjo4OXnzxRY4ePYqnpydvvfUWiYmJuLl9P82m7+dP/TXo7e2lvb2dkpISTp48SVxcHKmpqcyYMYOYmJjvvbdBQWGs2b9/P+Xl5VRVVREZGcnKlSvJzMzE399fbmkKExCdTsc777yDKIpMmzaNxYsXO+Vz7XY73d3dNDQ00NTURG1tLXFxcYSFheHj44MgCMyfPx9PT08E4VK1EBS+y1itVgYHB3nvvffYvXs3p0+fpqOjg7CwMJYuXcqaNWvklqig8L2hu7ubU6dOsX79empqavD09CQ1NVVas7+vjLuxKIoidrtd+v/h4WG0Wi12ux2LxYJarSYyMpLOzk70ej1GoxGVSkVoaCiBgYEEBASMt8SLYjQaqayspLi4mF27drFr1y7S09PJy8vDbDazZs0afH19ZdP3TRgaGqK/v5+hoSGSkpLw8PBwyQdAp9Oh1Wrp6ekBIDQ0lPDwcHx8fGRWNrFxPI92u53BwUGsVitqtZqQkBCXmAfDw8McPHiQL774gkOHDpGUlERMTAzR0dEuayyazWZEUcRqtWIwGGhra5NCVy5EQEAAoaGhBAUFyT7mVqsVo9FIZ2cn0dHR+Pn5yapnrNHpdNTW1vLhhx8SEBCAu7u7U4xFk8nEwMAARUVFlJSUUF5ezokTJ8jIyCAuLg5/f39UKhVZWVmEhYXh6ek57pomGna7HYPBgNlsRhAETCYTNpsNq9WKVqvFx8cHd3f3Czps3dzc8PX1xcvLy6XH1mKx0N/fT21tLR988AG1tbWYTCYiIiLIyspi/vz5zJ49W26Zo+js7ESn02EymcjIyHDZM8RERRRFBgYGpLnveMFIKKS3t7d0wzVRx91ut9PY2IjZbEalUpGcnIy7u7vsP49Op6OsrIyDBw+yd+9eMjMzycrKYtq0adKa/X1lXI1Fu93O8PAwBoNBeq+lpYWNGzdiMBjo6urC39+fX//61/zpT39i//79nDp1Ci8vL+655x7Wrl3LtddeO54SL6m9traWF198kd27dzM0NATAqVOnqKqqYtOmTcydO3fCGYulpaW8//77FBUV8dFHH5GYmOiSm2lBQQGbN2/mtddeQxRF7rnnHh566CHy8/PlljahMZlMmM1mDAYDu3btoq+vD39/f+644w7Z54Hdbqejo4NDhw5RUlKCIAg0NTXR1NREZ2cncXFxsuq7EDabjebmZqxWK93d3ZSUlPDkk08yPDx80Y1v+fLl3H333dx00024u7s7WfG/sdls9PT0cOrUKZ577jl++9vfsmTJEtn0jAeFhYV88sknlJaWsmbNGkJCQpzyuXV1dXzxxRc888wz6PV6yWF65swZ6WtUKhULFy4kPz+f2NhYp+iaSBiNRk6cOEF9fT0eHh6cOnUKnU5Hf38/GzduZNasWURGRhIUFDTq3wmCQEREBPn5+WRmZpKeni7TT3B5urq6KCws5J133qG0tBSLxUJaWhoPPPAA06ZNY/LkyS7lwBFFkTfeeIOdO3dSXV1NQUEBsbGxeHt7yy3tO4PZbGbbtm0cPXqUhoYGGhoaqK+vRxRF/P39mTp1Ku+99x6RkZF4eHjILfdr4yjg9Nhjj1FdXU1AQAD/93//R0xMjKw/j81mo7CwkOeff57m5mZ+8IMf8Jvf/IagoKDvbejpuYzJCNhsNrq6uqQDU0NDAyUlJQwNDdHS0kJxcbH0tVarlaGhIURRxM/Pj7CwMMxmM5s2bUKv1xMUFERaWhr5+fkkJiaOhbyvjUajoba2lp///OfU1NRIhqKvr6/kxdTr9ZSVlUk3o2M9mZqbm2lvb2dwcJAFCxaM2WJcVVWFIAhkZmaOyfcba0RRZMuWLaxfv559+/bJ7mn6LuC4OXrvvfeorq6mtbWVlpYWjEYjdrud0NBQWltbefTRR887eDkTQRAICQkhJyeHrq4uSktLZdNyOc6cOUNlZSVFRUV88cUX2Gw29Ho9vb29WCyWS87bgoICLBYLTU1NPPLII/j4+MjisTQYDOzdu5fHH38cd3d3Ojs70Wq1BAYGOl3LePCPf/yDbdu2UVhYiJubG3l5eSQlJTnlsxMTE1m+fDkvvPCC9Jx9FbvdzhNPPMFdd93F2rVrycvLc4q2iYDdbueNN95g8+bNlJeXIwgCw8PDUlSE2Wzm2LFjqNXq854dQRBQqVS89dZbBAUFkZSUxIoVK5gxYwbp6elERkbK9FOdj16vp7GxkYMHD2K1WgHw8PAgKiqK/Px8lzIUYcSQOX36NDU1Neh0Og4ePMiyZcuIj4+XW9qER6fTcejQIT755BN27NiBSqUiKCiI+Ph4oqKiqKuro7e3l6KiIn7yk5/w+OOPc91118kt+5JotVr0ej06nQ4Ab29vLBYLpaWlFBcXY7FYiI2NJTg4WFaDzGGrPProo/j7+7Nq1Sp+85vfEBIS8r2+TTyXb/3b0ev1tLW18c4772A0GtHpdPT19dHa2orFYkGj0dDa2nrBf5uVlcXUqVOJjo5mxYoVqNVqAgICiIuLIy8vz+mLuiMkpLy8nC1btlBTU4Ner0cURQRBIDQ0lIyMDLy8vNi8eTPvv/8+DQ0N5OfnM2fOnDHV0tnZSUVFBd3d3cyePXtMjEVRFOnp6UGj0WAymcZA5fjQ3t5Od3c3Go0GlUqFzWZDEATZDcfe3l46OztpamqipaUFjUYjhUgtWbIEf39/RFGkvb2d7OxswsPDZQudFEWRvr4+Kisrqauro7y8nMOHD9PV1YXBYGB4eBhPT0/pzwcOHOCWW27Bw8NDtlBfQRDw8vIiLCzM5Q2WQ4cOcfjwYY4dO0Z1dTUwsn6cG0VxsflqNBqpqqrC09OTm266iYSEBFk8qo7Ki52dnfj5+aHVahkaGnL5sb9SqqqqaG5uZmhoiISEBHJzc0lISHDKZ3t7exMVFcXSpUvZt28fXV1d2O12AgICMBqNWCwWANra2qiqqiIrK2vCGouDg4NYLBZMJhPV1dVUVFRgMBhQq9UkJyezYMECwsLCvvb3ddysOFIRANzd3fHy8rqgIzkwMBBRFNFqtZjNZgYGBqSXwWBApVIRHBzsEsaiKIr09/ezc+dODh06JDmk58+fz/z588nNzcXPz8/laiK4ubnh7e2Np6cnoijS29vL8PCw3LK+NtXV1fT09Ej7ZEtLC319fQCEhIQwd+5cZs2a5VRNmzZt4tChQ5SWlpKens706dNJSEggKioKgPr6esrLy9m4caPkqIyNjWXGjBlO1XkhHGfL06dP09vbK83nnp4eBgYGpLHNzMwkJCSEiooKBgcH8fPzk85Nl0rdGE8c0UGbNm3CbrezePFili1b9o3WLGdis9kwmUxUVVVRUVFBZ2endP7IyMggMzNzTPeUb20sGgwGmpubef3116UcqAvh6emJWq3GZrNhNptxc3MjNzeXlStX4uXlxdy5c/Hx8SEgIAB/f39iY2OdeoByhMBVVlayf/9+PvnkEzQazagJ7O3tzbRp0wgLC2PXrl1s2LBBWnDG2lhsb2/n1KlT9PX1jcli7Mipamtro7e3Fzc3N5eIET8XR3iCwxATRRE3Nzeio6MJCwuTLUTBcaiurKykrKyMwsJCiouLpUXRcUCJjIxEFEUqKyu54YYbyM3NdbqxKIqi5LSpqKhg27ZtHD9+nKNHjzI8PIxarSYwMJCsrCy8vb2pq6ujs7OT8vJyOjo6ZM8LVavVeHp6unx4TWlpKYWFhZSUlAAjut3c3AgODpacSw6Gh4cZHh4etTZ2dXVx6tQpurq6ZA+/gRGnnyNn3JUZHh7GZDJhMBgIDw+/4GHakevW3NyMVqvFy8tLCulzHLzGG8dztnbtWurr6xkYGMBkMhEUFITFYpGMRbPZjE6nY2BgwCm6xgqLxcLw8DBGo5GGhgYMBgMajYY9e/awdetWBgYG8PT0ZNGiRaSlpX3tg5coiuh0Ory8vIiOjpaeD29vbwICAi4YWhodHS1FN+l0OoqLi+ns7KSrq4uuri5SUlLIyspiypQpYzIG3wbHAXX79u2cPHkSPz8/QkNDWb16NcuXL3cJjRfCzc2N2NhYIiIi6OzsZGBgYEIYi4711+HUKCwspKamhuHhYZqamiguLqapqQmAhIQEdDodcXFxREdHj7s2URQxmUx8+umnnDp1CoDbb7+dtWvXkpCQIO3HXV1dHDhwgMLCQjo7Ozl+/DgREREuYSw65vPWrVuprq6mv78fGDEWtVotBoMBk8nEvHnzSEpKorW1FbPZTHR0tOzOG41GQ0VFBRs3biQrK4trr72WefPmyarpclgsFnQ6HW1tbXzxxRds27aNyspKNBoNAMuWLWPFihXExcURGho6Jrej39pY9Pf3Jzo6mpCQEAwGwwWNRbVazdSpU/Hz82NoaIjS0lLc3d2ZPn26y1yjDw0N8V//9V/s3LmTrq6uC35NdXW1lCeRn59PUVERvb29NDQ0jLmekpIS9u7dy9SpU8fk+zk20Y0bNxIcHMyyZctITk4ek+89Vmg0Go4fP84HH3xAV1cXHh4eJCUl8dZbb5GSkiJbSM7w8DBbt27l5ZdflvJmRFFk8uTJBAUFodFo+Pvf/y7NfUeC+t133+30MR4aGuLQoUN8/PHHfP755+c9kxERESxYsIA//vGPWCwWfv/73/Phhx/S399Pf3+/rDfOoigyODhIQ0PDRaMRXIW8vDyampokY9Hf35+MjAyWLl0qhRyqVCrsdjsnTpzg9OnTtLW1ySn5ssjl2f06VFVVUVhYyLZt23j99dcJDQ09L+9Tq9WyY8cOKioqsFqtTJ48maeeesrphxJPT09uv/129u7dS1dXF83Nzfj6+qLRaEYZ5fn5+axevdqp2r4tDQ0NlJeXs2fPHjZt2kRfXx8mk2mUk8TNzU1KR/m6qNVqXnnlFdrb2zGZTNKNsONmMSIi4pL/Xq/X88tf/pJNmzZJ+3lNTQ1Hjhxh+fLlX1vPWGM2m9m4cSNVVVW4u7tz9dVX8+c//5moqCiXz/9bvnw5er2ekpISmpqavtHv19lUVVXR1NREc3MzhYWFHD58mJaWllFf45i7ra2tvPLKK1KhtfHGZDJx6NAhysvLUalULF68mCeeeOK8A35kZCQzZszgrrvu4i9/+QsnTpzA09OTRx99dNw1Xg6LxcKRI0fYtGkTNTU10vvu7u5ER0ezfPlyDh48yPHjx9m/fz8wst9kZ2ezatUqWdNfNmzYwPbt26mqquK9994jLi4OLy8v2fRcCbW1tezbt493332XkydPnhd9t2fPHkpLS6msrOSFF14Yk1vSb20senl5ERcXx2OPPUZRURFubm4EBATw8ssvI4oiAQEBJCUl8corrxAYGIjJZKK2tpbPPvvMZeLcBwYGqKqq4sCBAwwMDCCKomTgTpkyBaPRyIYNGwgLCyMxMZG8vDwCAgIoLy8ftwPWWF/Ld3R08Oyzz6LRaEhKSnK5ggpNTU0UFRXx0ksvSfmv0dHR3HXXXSQnJxMUFOT02HG73U5fXx/V1dX8z//8Dy0tLXh6epKfn89dd93FlClTCAkJwWg08vTTT1NWVkZnZycAaWlphIeHO1UvwLFjx9i+fTtbtmxhaGgIm80GjBguMTExPPzwwyxfvpzo6GiXM8gcHtbe3l7JQwYjRbFqamqIi4sjMjLSJW7DHd7/e++9FxjJMwoKCiI6OnrUzaIjVOtvf/sbn3/+ueRxTU5OZvbs2UyePNllNiaH592Vef/999m1axft7e3U1dXh6elJcHDwqK/p7e3ljTfeoK2tjaVLl3LzzTeTkZEhWwGn1atX4+3tzfbt26UKgOdis9lcftwBGhsbee+996SCG21tbej1erRaLSqVivDwcBYvXkxKSgoZGRlMnjwZf3//b7zPx8TEEBERgd1ul24WryQdwWazcejQIWpra6XnzcPDg+nTp7Ns2bJvpGUs2b17N2+99RaFhYXY7Xbmzp3LI4884hIRBldCYmIikyZNwmq1snv3btasWUN6erpLhK/b7Xb6+/t56aWXqK2tlfaR/v5+BgcHGRoaQq/XExwcTHx8/KiCU+d+D7PZ7FTHqdVqJTIykpiYGJYuXXrROR4SEsKyZct45ZVXpDDEoaEhWds6nDp1ii+//JLXXnuN1tZWYmNjmTlzJoGBgSxZsoSsrCwiIyMpLS3lnXfeYc+ePeh0OoKCgpgyZQrz58+XRTeMGLmHDx+mqamJmTNnEh0d7TL78YVwRLG9/fbb7Nu3j+rqavz9/cnJySEjI4OpU6eyc+dOSktL6evro6ysDIPBgN1u/9bn529tLAqCgK+vr5SX4O7ujqenJ3v37qWhoYHQ0FDmzp1LVlYWPj4+WK1W6RDtCjdbGo2GkydPsm3bNnp7e7Farfj5+REdHc2tt95KYGAgPT09tLe3M3PmTObMmUNKSgphYWF4eXmh0Whobm6mp6dnzJJ0+/v70el0YxoSZjabqaurY3h4mJCQEKfl7lwp9fX1FBcXU1VVhcViISAggOTkZBYtWoS/v78suRuNjY0UFxezZ88e6uvrpcVt1apVLF68mJiYGEm7I0/GkdORk5PjlBCWr1JRUUFNTY10SPL19cXX15fg4GCuvfZaFixYQGpqKh4eHlLOpasgCALe3t5MmjSJqKgoyfCuqKjAz8+P2NhYwsLCXKIyWUhICL6+vlKFVpVKhbu7+wVvBRwHFofhDkhrjFxz+0JoNBpp3rgqfX199PX1SQe/r0ay6PV6Ojs7qa+vx2w24+PjI63VcjFlyhTUajW+vr6sW7fuvNC9hoYGysrKZCvodjnMZjPV1dUcOXKEHTt2oNFoGBoaQq1WM2vWLIKCgvDy8iIoKIg5c+YQFRVFXFwc8fHx3+pZ9fDw+FrGkyiKdHV1UV5ezmeffUZTU5NUbOr6669n/vz5TitwdDHa2tqkVIaOjg5yc3NJT08nPj5+wrSgOLcdSX9/v5TDL7exqNPp6Ojo4Msvv2TXrl20tbUxODgIjBgFVqsVURS58cYbiY+PR6VSSQ6P6upqmpubpZwvLy8vp6WQqNVq4uLiuPHGGwkLC2Pq1KkXnQfu7u6EhoZKrWT6+/s5ffo0U6dOlc3RoNfraWlp4cyZM7i7u5OcnMzq1asJDw8nJyeHiIgIqd2N0WjEarVKkYUZGRnnOfuchdVq5cSJE5w5cwZvb2+WLl3q8j1vHcVAjxw5QmdnJ5GRkfzwhz8kJSWFmJgYEhMTUalUDA4O0t/fT3d3N729vYSGhn7rzg1jcury8PAgJyeHlJQU1Go1w8PDzJs3j/7+fsLDw5kzZw5eXl6oVCo8PDyIjo7mxhtvHIuP/lY44qx3797Nm2++iclkkno8zpo1iwceeEAKXQgMDOTGG28kJCREOni4u7vT1tZGeXk51dXV5OXljclB9syZM3R3d6PX67/194J/5ys68uvCw8NJS0sbk+89FoiiSEVFBSdPnkSr1SIIAnFxceTm5jo9ydyhx2AwcPjwYTZv3syGDRvw9vYmKyuLa665hvvvv18KiW1ra6OoqIjKykr0ej2enp7ExsYya9YsWVo9tLa2otFo8PPzk/I9Y2JiSE9P55FHHiEyMhJvb2/pYHVuQRa5EQSBgIAAZs6cSUdHhxTiefLkSfR6PdnZ2cyePdsljEVHMZ5LGSGiKGI2mykrK6OtrW2U8ycwMJCoqCiX+VkcBQouFoLvCjjmqlqtvmjLke7ubhobG+nv75fySOWuZhcfH09oaCgpKSls375dKirlwFGswlVSMs7FMS927NghhTZFR0cTGxtLSkoKt912GykpKfj4+ODl5SVb/pGjuFRFRQXvvvsun3/+OUajETc3N/z8/Pj5z39Odna201qnXAhRFDl9+jSVlZXSjVZaWhqpqakXvPU2mUzY7XYpwsgVm4JbrVapvZHceZYdHR0UFBTwwgsv0NPTM+oZ8/HxwdvbGz8/P37zm98QHh6OIAjodDrpxry3t1cqzBQREeE0x4KHhwfp6emkpaVdcm2DEaekp6cn7u7uaLVaqTqqo9+lHDhu+0VRxNfXl8TERJYsWUJSUhKCIEipJdu2bZMuA4KCgli1ahXZ2dmyOUvNZjPbt2+np6eH9PR01q5d6yMRc/YAACAASURBVBL78aUwmUy8+eablJeX4+/vz4wZM3jmmWdGrR8+Pj6UlpZSUlIi9SqPi4tzDWPRgSMR12q1EhAQgFqtprm5mY8++oibb77ZpX4RjtL17733HocOHUKv1xMaGkpoaCh5eXnceOONBAQEkJeXR25uLnDh6oYWi4W+vj5ee+01/vCHP3zrX4goihw4cIDGxsYxSxzX6/V0d3dz5swZbDYbfn5+soRIXoz+/n727t3Lnj17UKvV+Pv7s3LlStkcChqNhhdffJE333yTrq4uvL29ufPOO7n//vvJzMwcVQCmpaWFv/3tb1LISmxsLM899xzJycmy5J788pe/ZMmSJRw7doxFixZJZakDAwPPOzT39fW5lLHo4KabbsJms7F+/Xq5pXwruru72bt3L+vXr+fUqVOjnuf58+fz4IMPyqhuxOjy8vIiICAArVZLd3e3dJvrahgMBl544QUOHjyIRqMhJSWF7OxsAgICRn3N7t27+fTTTzEYDEybNo1FixaxdOlSGZXDgQMHqK6upq6ujsrKyvPW9cTERLKzs2VSd2ksFgsvv/wyGzduxGQy8eCDD/Lkk08SEhIiFUiT24CxWCwUFBSwa9cuCgsL2bdvn1QcLTMzk4cffpgpU6bIVpkaRkIbBwcHpTw/B76+voSEhJxXeEkURd5//306Ojokp/FTTz0l2y3MpbBarRctbOhMjh49yjvvvHNebriHhwf33nsvs2bNIiMjg+zsbGkvDAsL44033qCsrEyquhsdHc3tt9/OAw884DTtVxoi7+XlRUJCAqmpqVitVsxmM42NjbKGsScmJnLttdeyc+dOmpqaOHjwIM8//zx/+tOfsFqtlJeX8+KLL7Jz507c3NxISUnh/vvv595775U1V9FisbB//36ioqKYMWOG7M6OK8HhpFapVERFRbF8+fKLGttubm54eXkREhIyJmfRcbHevLy8uOOOO2hra+PYsWOUlJRw4sQJJk+e7BKLnSOu/X//93/ZsWMHXV1dhIWF8atf/YrU1FRCQ0MJCwuTFpTLbYZWq5WOjo4xeWBFUZRCIzw8PEhMTPzWRvbBgwfZvHkzAwMDhIeHExYW5hJJ9CaTiZ6eHp577jlKSkqkXK81a9awaNEisrKynK6ptLSU/fv38/7776PT6YiJiWHKlCk89NBDJCYmSjdJFotFCp3t7OzEarWSn5/PkiVLWLBggWxhb8HBwVIz6qCgIMkL+VVDURRFqqurXbIK47meSsd/Jwp2u50DBw6wf/9+KioqqK2tHRUOBSPV9hwh+3LiuAnKzs7myJEjsmq5FEajkba2NjZt2kRvby+ZmZk8+OCDoyoki6LIunXr2Lp1KyUlJYSFhfHss8+OWbTH10Wv19Pa2srzzz9PVVUVAwMDDA0NXdABmJKSIjkkXQGz2cyePXsoKyujoqKCxYsX8+CDD0rpJo6iQnIbifDvnshPPvmk1G7p3PXCw8ODyMhI2fU6ej82NzfT1dUlrW3JycmjUkKKiopobW1FFEU2bNgg9Vp2/N2Pf/xjbrnlFpc4R7kS5eXlFBcXU1tbK73n4+NDREQEjz76KAsWLCA6OvqCPW17enpGOU0d/TjlDqu9EIIg4ObmhoeHh2QkyL0/BgYGkpaWxuTJk6VWGceOHeP48ePs2rWLo0ePSs7SefPmsXz5cm6++WZZnTeOtmKVlZU8/vjjXH311aP+Xq/XU19fT0BAAGFhYbJqvRAOZ5ifn590YdXW1sbRo0elc0dWVhYPPPCAFPnxbRmXXVStVpOWlsaCBQsYGhpi586d7Nq1i4GBAWJjY6WiN3JdPxsMBjo6Ojh48CAtLS2EhYUxe/Zsli1bRmxsLJ6enlJ1oSvBbrczPDw8Zg+to4Klp6fnN8r7cCQ+O/pd7tu3jyNHjmA2m5k6dSqTJk2SrdjDuQwNDdHQ0MD+/fvp7e3F09OT8PBwFi1aRHp6utMfUKvVSk1NDXv37qW5uZmoqCjy8vJYuXKlFOZhs9kYHBykqKiIsrIyTp48iclkIjY2lvz8fBYtWiRrfx53d3eCg4NHHSYGBwfp6OgYlTPnCMHW6/VSywdXcSJMBEwmExqNhp6enlF94Gw2G9u2bZOS5gcGBrBarVIPWUf/rPT0dNlzFR03i3IUj7pS9Ho9dXV1FBYW0tjYKOX0zJs3Dy8vLwRBwGAw0NTUxO7du6msrMRkMjF79mxmzJghW1ikwWCgpaWFPXv20NXVdcliGQaDQWpaLTeOm4CdO3dy+vRptFotDz74oJTDlZycLLuT41z6+vokQ+Grzlq73Y5Go+HIkSNSaGFISAgRERH4+/s7dc7bbDZ6enpoamqS1gtfX99RPWW1Wi3V1dWUlpbS29srlcK3WCx4eXlx5swZKQR46dKl0g2DwkiqQl1dnZQf7mhmn5eXx/Lly4mPjx91YLbZbBiNRnp6emhtbR21D06ePJno6GiXioQ7F1EUsdvt2O12yXCU0xHi6elJUFCQ5ADVaDS0tbWxdetWDh06RF1dHVqtloSEBKZPn87s2bOZNGmSbHphZN2oqKhAFEVSU1NJTExEFEXOnDlDQ0MDLS0t1NfXExgYyNSpU8nIyJBdM4ysaVqtFrvdztDQELW1tezevZuOjg7OnDlDaWmplCsaFxfHokWLCAwMHJOzxrg9DZ6enlx//fUIgkBBQQHr1q3j8OHDJCcnk56ezn333Yevr68sB6be3l6qqqo4efKklKT/2GOPuZR3F0a8ohfanM+tlGqz2aQ/i6KIzWbDYDDQ3d3N8ePH2bhxI2VlZVKOxKpVq8jNzZXdKLDb7fT09HDs2DGampqwWq2EhYUxc+ZMVq5cednS6OOBwWDg9OnTHDhwAEEQyMnJ4YYbbuCee+6RQj4GBwdpbm7m97//PdXV1Wg0GlQqFbNmzeKaa65h0aJFTtd9Mex2u5RTsmPHDgwGwyiHRm1tLYODg/j4+EjVtEJDQ2VUPBrHBugKNxjn4ujJWl5eTmFhIQUFBaOex+PHj5/nbPL09CQhIYGHHnqIBQsWSLcdrsS5+VGugNVqpbGxkS1btvDmm2+i1WpZvHgxc+bMITk5WRrf7u5uPv/8c/bt24fBYCA+Pp5bbrnlW6cEfBsMBgNdXV1oNJrLphOcOnWKgwcPukRvL7PZzCeffMKnn36KIAgsXbqUadOm4evr65KGSXd3t5Tb/NUoBLvdTkNDA3/84x+ZOnUqubm55OTksHDhQrKzs/H29nba+WN4eJiKigoqKyvp7u7G3d2diIgIgoKC8Pb2xmazUVdXx+nTpykqKuLIkSNSsbTAwEDCwsKoqqri8OHDmEwmcnJyiImJccnfiRzs37+fmpoabDYbarWa9PR0Fi9ezNq1a8nMzDzv64eGhmhra6OgoIDy8nI0Gg0+Pj5MnTqVGTNmSMXrXA3HxYTZbMZutyMIAn5+fuftkQ5j0nEDNZ57qCAIuLu74+PjI9Us6enp4aWXXpJu0D08PKSoq8mTJ4+bliultbWVw4cPEx8fT3R0NIGBgdhsNnbv3s0HH3xAeXk5RqMRtVrNqlWrWLVqFbfffrvsDl673U5bWxtms5n29nY2btwoFfPSaDS4ubnx+uuvs2DBgjEvIDqurpPIyEiuueYavLy8eOKJJzhw4AD79u1DrVaj1+tZvHgx2dnZTj+g7t27l9deew1RFMnJyWHu3LnfqJCKw2gb6zYXDqxWKwMDA1LfNgf19fVSkYTCwkIpjMJsNvP5559LHlZ3d3dycnKkpGmr1UpWVpbTGlNfiiNHjrBt2zZeffVVrFYrqampLFq0iKeffpro6GhZNkGLxYJer5e8k4888gjTpk2jtbWVDRs2cPjwYerr62lpaRn1e1GpVFx//fVkZWW5RNllq9WK0WiktraW9evXc/jwYYqLi4F/z1mVSiXlr06bNo2//e1vUtK/wsWxWCxUV1fzq1/9irKyMnp7e897Ps+9wXWQlJTEokWLuOOOO1Cr1S45zgaDYVS4rFw4GrJv3ryZZ599VsrfFgSBmpoatm/fjlar5YEHHsBsNlNTU8PmzZsxmUxMnz6dq6++mjvvvFPWmwFHqHF3dzdvvPEG9fX1F01T0Gq1LlOF1mAw8PLLL2M2m7nqqqu4/vrrLxi65yokJiayevVqamtrqaqqkirlfpXy8nIqKyulokd33nkn1157LVddddW4F70ZHh5Gq9XS3NyM1WrFx8eH8PBwbrnlFnJzc/Hx8aGmpobXXnuNPXv20N7eTlhYGFdddRXXXHMNkydPxmg0cuedd0oOS4XR/PCHP8THx4fCwkL8/f159tlnyc7Ovmi47r59+9i+fTvvvPMOw8PDREdHM2vWLF566SWioqJczpHnwGQycebMGekWNTIyksWLF0vOf7vdzsDAALW1tTQ3N9PR0cGqVauYNGnSuF4QqFQq4uLiLhixFhoayuzZs/nZz35GSkqKrEWmHNTW1rJjxw7+/Oc/k5KSgkqlQq/X89e//pU1a9bwn//5n+Tm5rJv3z7Wr1/Pn//8ZyIjI2VNMYIRwzw4OFhqAVNaWirdNKvVary9vUlJSRmXXNBx3U0FQSAiIoLFixfzu9/9ji+//JLy8nIaGxv5+OOP6ejoYN68eVJTTmdsSKdOnaK8vFxKMp86dSpJSUnf6LMduVWO3MKxqkbl7u6OWq2mr6+Pf/zjHxw8eHDUQ1hTUyOVIO7v70cURSmRdebMmURGRhIVFUVSUhIJCQm89dZbdHV1YTabiY6OHlUUwtk4+s6999577Nu3D71ej81mk3rURUREyHYwcXhy/f39GRwc5IUXXiAwMBBRFGlpaaGvr4/h4WFUKhUBAQFS/xpHBU+5Qt4c2Gw2qTrrjh07aG1tpbW1lcHBQfz9/UlNTZVCIx23HaIoMjQ0RElJCTExMQQEBLikIeMqOIxtk8mE0Wi84K3RhRxHLS0tlJSUUF5ezuTJk12yn1pdXR3V1dVyy+DMmTOcPHmSV155hba2NiwWCyqVioiICLq7u9HpdKNyQVtbW6mpqcHLy4sVK1Zw6623yh5C5mgDc9111+Hl5UVLS4tUDddut9Pa2sr27dul8FRXudFVq9UkJCRw5swZGhsbef/994mNjSU1NdUlc7iCg4PJy8vj6aeflipB63Q6dDodNTU11NXVUVxcLFUEd0SI7Nixg46ODqqqqnjiiSfGdc3r7e2ltLSUdevW0dfXx6RJk1iyZAk//vGP8ff3p7Kyks8++4w9e/bQ399PWFgYs2bN4j/+4z8IDg6mv7+ft99+G5vNRmxsLJmZmYSHh8s+x12JmTNnEhcXxw9+8APc3NyYPHkyAQEB550l7HY7paWlbNmyhX379jE8PIxarSY7O3tUxIer7YFWq5XBwUF6enqoqqqSCgr19fWxbt06jh8/jsFgkHoUO0JrVSoVRqORH/3oR+PaLs1isVBVVXVe9FJSUhJz5szh4YcfJi0tTdZoDwddXV1StVxvb29UKhVDQ0M0NzeTmZnJvHnzmDFjBsHBwSxYsICWlhYKCwvZsGED06dPl9VY9Pf35y9/+Qu1tbX09vZKxTVtNhuBgYFMnz6dpKSkcUnhGvfVxtE3bfXq1ahUKoKDg/Hx8aG6uhq1Wo3JZCIiIoIFCxY4JTSys7OT7u5utFotnp6eZGVlfa1YZEc5/JaWFmw2Gz4+PkRGRjJr1qwxSSIVBIGsrCyampoYHh6Wevide/3d2toqVQ318/MjODiYoKAgIiMjSUtLIzY2lujoaJKTkzEYDPj6+iKKomQIyXlQtVqtVFZWcvTo0VEH0/j4eJKTk2XNpXR3dyc+Pp7c3FyKioo4dOjQqIpSjnAFxwbf0dEhaY+JiRmT3/83xWq10tvby4EDB9i5cyc7duwgMDCQkJAQ4uPjCQkJISUlhV27dmEymaRbDpvNRl9fH/v27SMnJ4f4+Hj8/f1dYrM8t8DNuY4ROVGpVFITXFEUaW9vJyQkZNR4OXSbzWYp8Vyr1dLR0UFHR8cFw6LkwsPDQ/JC9vf309fXJ6ue3t5eiouL2blzJydOnECtVhMTE0NcXByxsbGUlZXR3d1NR0cHRqMRvV7P0NAQOp2OgIAALBYLRqMRo9Eo5TTKhVqtJjU1FZvNRn9/v2QY2u12qqurqamp4fTp05hMJvR6PSaTyel9vkRRlELRHflPS5cu5fjx43R3d1NQUMDOnTvx8PAgLS3NJSInzsXRviYiIkLq/2gwGNBqtZw+fZqysjKsVqtkrDv6yjY2NmI0GjEYDNx0003ExsaO277Y399PfX09paWlwMh+sXTpUiZPnsyJEycoKChg7969tLS0kJ6eTnZ2NgsXLiQrK4uqqiqKioooKCggNDSUadOmMWfOHFn3mq/i5eUle1pLVFTUFUVM2Ww2vvzyS44ePSpdGKSnp5Ofn8/MmTNlO3/Y7XYsFgtmsxmj0cjg4CA6nU7qDWk2m6XWRo5+2TBSj2D//v10dnZKUVGO856Hh4eUrzbezpCqqipOnz59Xru34OBgkpOTXSLE3sHAwICk05EK5wjbnTdvHunp6VK3gNjYWHJzc+nt7WXHjh2XzD13Bu7u7ixfvpyMjAy6u7tpaWnh9ddfB0bydOfPny9Vqh5rnOKaEgSBhIQE7rzzTpYvX05BQQE///nPpRu+5uZmcnJynLLgDA8PMzw8jN1ul67Hv07PQavVSnt7O2+++aZULXPu3Lncd999Y2YsPvDAAyQlJXHo0CHKysrO+5ro6GgSExPJy8sjPT2drKysC94WiqLI1q1bqa6uxmw2M2nSJNljrk0mEx9//DE9PT2SFkEQWLBgAUuWLJFVm4+PDytXriQ8PJyf/vSnaDQa/P39iY2NZcWKFVxzzTUkJCTg5ubGAw88wMDAAD4+PixevFjWsBWH1/Hw4cP89re/pampCX9/f+644w5pjqSlpdHX14dGo6GrqwudTodKpcJkMtHS0sKbb7456pDiKs1pHYbX4OAg5eXlspdod3NzIzExkaeeeopTp05RWlrKokWLLjheHR0dbNu2jTfeeAOj0egS4/lVgoODmTZtGv/617/klgLA4cOH+eCDD9i4cSPu7u4EBgayfPly7rnnHgIDA3nttdcoLCzk1KlT1NbWSs4DR8+01157jSNHjvDGG29866bwY0VGRsZ57yUmJtLa2kp9fT29vb00NjbS1dVFbGys0zSLoojRaOTUqVNkZ2fj5+eHr68vzz33HJ999hk7duxgw4YN/OEPf8DPz4/AwEDi4+Odou2bEBQUNCr8aubMmXR1dTFt2jTWrVvH6dOnR7WG6ezsZHh4mE8++YR77rln3PLkNRoNLS0twIizKTMzk5tuugmAd999ly+//JK6ujrUajW33XYb119/PVOmTOHo0aO8+uqrfP7556jVau69915uuumm8yo3yk1UVBTR0dFyy7giTCYTzz77LENDQ8DIen7fffdxzTXXyNK+xhFCaDQa6e/vp729ncbGRoqLiykuLpbC0x0XFF8thGWxWGhra8Nut0sRT6tWrWL+/PmEh4cTGBg4roVZRFHk2LFjvPrqqxw4cEBKcXH8nUqlcrl9T6fTYTAYpNBZLy8vqT7IpEmTpB7aDnJyctDpdLzwwguytig5l/j4eAIDA6W8VRhJ+7vtttvGzeHh1J3Ux8eHpKQkoqKiePLJJxkcHESv13PkyBEKCwvJz893WsUhQRDw9PQkNDT0iq9sLRYLO3fu5IsvvmDdunX4+fmxZMkSbrrpJnx9fcf0oVi0aBHz5s27YP4TjGw6bm5uqNXqixqAoihSVFRER0cHAQEBrF69WlaPZH9/P1VVVezdu1fKC1Sr1VIRBVdIKo+OjmblypVSpSxHyXMPDw/c3d0ZGBigvLycw4cPYzAYSEtLIyEhQTYj3Gq1cvDgQbZs2cK6desYGhpi0qRJzJgxg/vuu0/K2ejo6OD++++XNiAPDw/uv/9+tFotTU1NFBQU8N///d+kpKSQl5fH3Xffjb+/v5TjI0cbE/h3cZu+vj62bdtGZWUlGRkZsvZngpGFOTQ0lKuuuuqiYUspKSlkZWWxfv16KQTR1fD29iY2NhZBELDb7VLLB7mKZ3z66ac0NTWRnZ3Nvffey+rVq4mMjJRuCX/xi18wbdo03nrrLUpLS89zHqSkpJCfn09iYqLL5tlZrVYaGhp48cUXMZlMBAYGSmu5sw5Wvb29fPTRR3zyySe0trbyi1/8goULF5KXl0dQUBBr167FbDazYcMG9Ho9g4OD0gF7IhEREcHNN9/MqlWrePXVV9mwYYN0wwf/bio/Vj2NL4Qjz1AURebOnUtWVhY2m43Tp0/T2NhIb28v7u7u/PrXv+aGG24gMTGR2tpaHn/8caqrq4mOjuahhx7ivvvuk7XS9sUIDw93iToIl6OyspKPP/4Ys9ks7e0As2bNkkW/2Wzm4MGD7Nq1i/3799PU1ITJZJJuGW02GyqVCm9vb3x8fCSD0JEjP3v2bObOnct1111Hdna2VBXV09MTNze3cTXUHNV9t2/fzvbt29m5cyeiKJKSkkJsbCxRUVF8+umn6HQ6tFrtuGj4tjiiVjw9PVGr1ZKR+NUxcxSYcjV0Oh3V1dVOK0znFGNRFEU6Ozupr6+ns7OTjo4OKbbZw8ODsLAw4uPjnXoA9PT0ZOrUqVdk5JnNZqnB9p49e6ioqCA1NZX77ruPadOmkZqaOuYPpbu7+5jcVjluUX18fJg1a5asYZ4lJSWsW7dO8uj6+voSExPDvffeKyUZy43DMLxYErbDG+8IDwkMDCQ/P1+WGwxRFNm9ezc7duxgz549mEwmrr76aqZPn05+fj6hoaEMDAxQXV3N7t27KSsrQxRFsrKyWL58Oddffz1Go5HOzk6mTZtGWVmZVGreYVA6evH98Y9/dOrvJzw8nPnz53PkyBFsNht2ux2z2SwVMJDbWHRUgLvUMzo8PExTU9NFHT6ugMPp5DAWRVEc14Pz5UhOTiYpKYnY2Fiuuuqq84omOMJ7e3t7EUWRJUuWkJWVJRVJS0xMJC0tzSXWkotx/PhxDh06JIU0JSUlMX36dIKDg52m++9//zsHDhygra2NBQsWMH36dGJjY4GRMa6srKSmpgZBEEhPT2fSpEkuUZji6+I4PHt4eDBp0iTi4uJGGYsqlYrQ0NBxXb/PLYDn7u6Om5ubVLn83IOeRqPhgw8+wGazSSG1a9euldoNOPpbuhoqlcqlnzf4dxuVmpoayVCMjIxkyZIlJCQkyJJL9+GHH3LgwAFOnDhBZ2endHERGBhIdnY2arUaHx8fqa3Vhx9+yLFjxxgYGCAkJIQFCxZw3XXXkZWVRUhIiFN/BxaLhdOnT7N582apeN7atWtZvHgxfn5+dHZ28vnnnxMUFORyPUHP7R5w7phd7Azf29tLe3u77GkNX8VRM0UQBKKjo4mLixvXS4txPeFaLBYMBgMajYbi4mKpF05jY6O0UTpKSMfExDi1r54jf/Jyg6vVauns7KSkpITPPvtMsuTXrl3Lj370owmzgXp5eZGUlCRrWFZDQwNbt26V4sUdPWyuvvpql2t6ejHc3NxGbSw+Pj6kpqY6/WbRarXS19fHF198wf79+6mtrSU1NZVrrrmG3NxcUlJSaG9vp6KigqKiIjZu3IggCKSlpZGXl8dtt93GtGnTsNvt6HQ6UlJSiImJ4fjx41RWVrJv3z5g5AYtJyfHqT8bQEhICHl5eRw7dkwytkRRpLq6+htVLh4LHLmeFypV/lX0ej1nzpyhoKBA9tDZS+Hh4UFwcDBeXl5Srl97ezvx8fGyHABzc3MJDw8nNTX1PG+uo91OZ2cnfX19uLm5sXDhQqk/LowUAHD2wc9xuB8cHCQwMJCAgICLrgdWq5Xjx49z5MgR6b2kpCTy8vKcqvvIkSNUV1djs9mIiorCYrHQ2tpKW1sb7u7u7Nu3j7KyMry9vZk1axaJiYkuVeDGUSkckHKzLoXDaPxqqou7uzuxsbHjaoR5eXlJhcP0ej06nY6hoSGGhoYkp6MoitTV1VFfX8/AwACiKDJjxgyuvfZali5dio+Pj0saijDaGHZFRFFkYGCA9vZ2KU/R39+flJQUrr/+esLDw2Vxom/evJmTJ0/S19dHSkqK1Bc2IiKCefPmSe0ogoKCsNls7N+/n/Lycjw8PMjJyWHOnDnk5+c7Xfvw8DB9fX0cPnyYoqIijEYjGRkZrF27loULFzI4OMj27duBkTOeqxmL8PUKirW2ttLY2EhUVJTsaVwOhoeHpVx9Nzc3UlJSSEtLG9dUvnG1HBxG1tatW9m2bRs9PT1SgjkgtXQICwtzmhHj8OQZDAapGuelOHjwIF988QXr169Hp9Mxa9YsVqxYwTPPPOMUvWOFKyzmJpMJrVYrhXZOmjSJG2+8UQp3nAiEhoZKN4nnzmVno9Fo+Ne//sUHH3yATqcjKiqKp59+mrlz52IwGDh27BgfffQRRUVFdHd3A/DEE0+wYsUKqZqeg7CwMMLCwpg3bx7l5eWScQkwf/58br31VqcbDr6+viQkJJxnlMk1jx09/zo6Opg7d+5lvYxHjx7lyy+/5O9//zsGg8GJSr8e4eHhLFy4kKSkJOrr62ltbeXjjz8mLy9PFsfS2rVrL/i+oxDL0aNHKS4uZmhoiOTkZGbPns38+fOdrHI0g4ODUjXLVatWsXr16gsekByVoA8cOMCBAwek9zMzM53uAFm7di12u539+/fz/PPP8/zzzwMjzrCkpCRaW1ulNfpnP/uZy7QFgn+HwP3rX/9CFEUSEhJYu3btJZ9Hm81GV1eX1G8YkEL8pkyZMq4/W2JiIvn5+cDIrXJCQgKzZs2ioqKC3t5ezGYzVquV7du3S9FWwcHB3H333dKNoivjqDLrqhiNRgoKCvjyyy8lJ01ubi7Lly9n5cqVskVbFRcXo9VqmTJlCq+++iqZ/YztJAAAEbxJREFUmZnnHfYdkUx/+MMfOHHiBENDQ4SGhvK73/2OKVOmyKK9vb2dI0eO8MILL6DX61m5ciWPPPIICxcupL+/n8LCQtatW4fFYiEkJMTl5u+5N3KXw2q1cuLECY4fP86aNWtkL+TkoLOzk5MnT7Jlyxb8/PxYsGABK1asGNfc4TE/DZhMJunGw1FcpbW1FYPBMCocKzs7m2XLljF//nwWLlzotJhgRxy3zWaju7ubV155hXnz5jFz5kwCAwNHNcatq6ujpaUFg8FAQEAAc+fOZdmyZVx11VVO0TpWnNucferUqbIsMH/961/ZvHnzqOblwcHB5ObmunwIy7n09PRQXFws++bY3d3Nyy+/jFarRRRFNBoNr7/+Ov/4xz/QaDR0dnYyNDSE2WyWPKgPP/wwgYGBl6z6l5mZSWJiItdddx0w4hWXI0QnKSmJn/zkJ7z55pu0trYyNDSEzWbjn//8JykpKaSmpjolz8RgMFBRUcH7779PcXGx1Ms0IiLiPGPKYrEwODhIQUEBb7/9NkVFRaMiKByVG10plAVGnHZXXXUVJpOJuLg4nnrqKZfZFB2YTCY++OAD1q9fT1NTE7Gxsbz99tsu0eD5pZdeYufOnVRUVNDQ0ICPjw9paWkEBwcTFxcnFa9pb2/n7bff5vjx41IO6x133MHs2bOdHqHyk5/8hEmTJpGTk0NNTQ0lJSWS41Sn05GTk8Ps2bO55ZZbmDp1qssYigaDgYaGBh5//HEqKiq44YYbuPXWWy/6TFksFim889NPPx0VghoaGkp6evq4p2c4Uht8fHwwmUzs2LGDwsJCLBaLVPHSgaenJzExMSxatIglS5ZMiMiliooKUlJSXK7IjaOC9rPPPitVm/X09OSxxx5jzZo1ZGVlyV7528vLi+DgYDIyMs57xiwWC8eOHeP/b+/eYpo+3wCOfwsVSktbDgWkQk+cihSpWBAQFhGmoG7uIDEuy2KybJm7WLKrxZslS3YwS3a3i2XbzeYOmdvEQ9yIp4lbUCaecIpocHbCCoocnRQo9H9hfr+/E3dgQvtD38+NJqA81vb9vYfnfZ4PP/yQPXv2MDo6Sk5ODlu2bMHj8YSt/oTX66W5uZmhoSFcLhfl5eVUVFQQGRnJ6dOnOX36NH/88QfBYBCtVqu4rDGLxYLVaiU2NpZdu3axatWqKXOJYDDI+Pg4W7du5fjx46jVal555ZWwZ1dIrc7eeustGhsb6enp4aWXXmL9+vWzXqBpRhaL0kmdVMBEKvUsNcmVLsYnJiaSmpqKy+WioqKC3Nxc7HZ72PrTBQIBfvrpJ3w+H62trRgMBi5fvkxnZyder5dr164RExNDUlISWVlZ1NXV4XQ6SUtLC0u8/5V0KXpsbCzkJzPSLvDJkyflFBC1Wo3T6cTlcpGYmKi4yfPfkdL0wk3aAJB+7/f75f5LGo1GLrNutVrJzs5m6dKlmEymfzzBle7hhXuAV6vVGI1GUlNT/zSGDA8P4/f7Q7ZYl05wjxw5wtWrV4mKiqKxsZH4+Pgpi0Wp8fbRo0flQkKTk5NYLBaysrIoLCzE4XAookLn3WJiYuQFbTh7nd6LlPZ0+PBhfD4fOp2OoqIinE5n2N+nAFarlQULFnDlyhUuXbok9yZMSkoiIyOD8+fP093dTV9fHydOnGBgYIB58+aRkJDAmjVryM7ODnk7I71eT0FBAXFxcVy/fp3Ozs4/9V41mUxYrVYWLlwo9yJTgvb2dg4dOsSZM2fo6+tjeHhYPpm7s0BQIBDgwoULXLx4kZaWFpqamrh06ZKcDRIVFUVeXh4rV66c9YWwwWDAbrdTV1fHiRMn8Pl88jNEpVIRFxeHw+EgKSmJpKQkrFYrZWVlxMXFKTLjJiYmBoPBQFJSkpwtFs4sm78i3W0+cuQIXq+X8fFxTCYTtbW1OJ3OsKdHxsfH09XVxdWrV2loaJD7/hmNRoxGI62trRw/fpyjR4+i0+nkoodSm7ZwfSYDgYBciEeqaxAZGcmFCxdoaGjg559/JhgM4na7KSkpCUuV2b9jMBjkdin19fVotVrcbjcWi4X+/n4GBgbw+XycPn2alpYWHA4HbrebBQsWhP3zODY2RkNDA62trdy8eZPs7Gxqamqw2WyzPo7d16xlcnJSzpvt6enh8uXL/Pjjj+zevRuv1wvc3rXWaDQYjUacTieLFi3iscceo6KiIiwnXNHR0XKp3LGxMXk3+OjRo2i1Wq5du8bIyAhqtRqtVkt2djbZ2dm4XC6eeuopxeywTpdUnjkci8Xff/9dbiKqUqmIioqitLSU4uLiKWWKlW5iYiLsvXbg9gM7KytLvhcspQLpdDosFgt5eXksXrxYblackJAQ9oFuuiIiIsjIyODXX3/lxo0bcmGCUG4uDA4O0tDQQHt7O2NjY+j1er7//vt7PqyvX79OW1sbbW1tcuEYg8FAcXEx5eXleDwebDZbyGKfLqmfpd/vn/Hqzvejv7+fc+fOcfz4cQKBAJmZmVRVVREXF6eIhbfH46G7u5uuri7a29vZvXs3MTExJCYmYrfbaW9vp7+/X55Qq1QqTCYThYWFLF++/F9t4syG9PT0kFUfnyler5empiZ6enqA21WSf/nlF0ZHR9FoNPJncmRkhMOHD9PY2EhDQ8OU6yZms5mSkhJWrlw56zHHxsaSmZnJc889h16vp62tTd44BUhLS6OkpITMzEwsFgtpaWmK6sV6t5iYGOLi4jCZTPT29jI2Nqa4xWIgEGBgYACv18v58+dRqVQkJSXhcrnCcs/vXlwul7wR9uWXXzI8PIzBYMBsNrNgwQIOHjyI1+vF7/fj8XjYuHEjxcXFWK3WsMYt3VFVqVRERkZy69YtvF4vjY2N7Nu3j+7ubtLT01m5cqXct1BJtFotOTk5VFZW8vbbb5Oamsrw8DCFhYV0dnbK89WDBw+SkpJCSUkJq1atCnu2zcTEBIODg+zcuROv14ter5cr4oZis/G+nrTDw8Ps3LmTHTt2cP78eTo7O6cMGvHx8bhcLjZv3kxRURFmszmsTeHz8vIoKCjg7Nmzcg/DW7ducevWLXlyJDWqfvLJJ3nxxRdxu91hi3emjIyMcOrUKerq6kL6c6UCKlIj1MjISJKTk9m0aROlpaUhjWUmJCQkUFhYCIT3Yr/dbuerr75i3759dHR00NPTg8lk4oknniA1NRWdTqfYggj/VmRkJHV1dVy8ePFPd43CaXh4mM8///xvvycYDBIbG0t6ejrLli3jzTffVOwpwZ38fj9dXV0cOHCA1atXh3WcvtP+/ft5/fXX+e233+Qqvps3bw53WDKXy0VXVxcDAwM888wzvPPOO3Lq9L3etzqdjmXLllFfXx+GaOc2qS3Qjh07ANi7dy/79u0jLy8PvV4vVxodGhri8uXLcosmSUREhNxLsrS0FLvdHpK4NRoNy5cvZ/ny5SH5ebMpMjKS6Ohoubdse3s7drudDRs2hDs0WWdnJ8eOHWPv3r0Eg0HMZjO1tbVs2bJFMePap59+yqFDh6ivr+fjjz+WW2JIysrKePzxx6murmbFihWKeX6MjIzI7TCMRiNNTU3s2bOH5uZmgsEgFRUVvPbaa6xatUoxMd9NqvA8MTHBBx98IN+xhNuf1YyMDJ5++mleffVV9Hq9IjIrfD4fzc3N7Nq1i2AwKGdGhOo1nvZiUVp17969W94h8/l8+P3+P5Vcj46Oprq6moqKClavXk1aWpoiKnqZTCbWr19Pfn4+9fX1fPvtt/T29gK3J3mPPvooRUVFlJeXk5eXp8j+KtMVqj4sDwO9Xk9WVhYWi4Wuri5GR0fp7Oz8V5V1Z5I06ZE+Y4FAQE7d/Ku+f3NNREQEdrsdh8NBR0dHWNJ/zWYzb7zxBt988w0tLS1cvHjxnt+XnJxMTEwM0dHRFBQUUF5ezsKFC8nOzlb8QnFiYoL9+/fT29vLwoULsdvtiorXZrNRU1PDRx99REFBgeLSmuD2xC4/Px+VSsXg4CDXrl0jEAjQ0dHBsWPH5MqdOp2Ol19+mTVr1oQ54rkpLS2NsrIyampqaGpqYmhoiPHxcdrb2+U0VKktxZ0NtNVqNR6Ph6KiIkpKSqiurg57+525TK/Xk5OTI5/qKu1k8dtvv+XgwYM0NzczOTlJZWUlK1asIDU1VVHPxuLiYnJycnjhhRemfE2r1aLT6RRXAFC6iw1w7NgxuQaIXq+nsrKSqqoqiouLFRXzvWg0GjZs2EBVVZXcexP+XznZaDQqZqEo9bR8//33mZiY4Pnnn6e6uppHHnlE2YvFkydPcujQIbq6uhgeHiYQCMh5wFJDUKnXmzRhUspujlqtZv78+cTExDAxMYHJZGJoaEj+utvtJisrC6fT+UAsFC0WCwaD4U//xlCSWpSkpKTQ09Pzj9VnlS4iIgKtVktZWRk//PAD/f39NDY2kpKSQkJCQkjTWyIiIsJ+4Xo2Sel6Uk5+X18fcLuJcqjSl7VaLUuWLGF8fJykpCQSExM5efKk3EIjJSUFs9ksV5iNjY3FZrPhdDoxm82KqwR3LyqVCp1Oh9FoJC4u7l+1Bgklq9Uqn3RWVlYqMpVXr9ej1+sJBAKsWLGCmzdvMjk5ic/nw+12y+OeRqOhpqaG3NzcMEc8NxkMBjIyMqirqyMuLo7Ozk4GBwcxGAz4fD5GRkaIiIiQF4LR0dEkJycTHx+Px+PB5XKRk5NDQkKCIlKY5yqpD25TUxNWq1UxxW2kfopdXV10d3dz8+ZNXC4Xbrcbh8MR9sOKu8XGxhIbGyu3/pkL5s+fT2FhIWVlZZw7d45gMEhiYiJOp5Pa2lo8Hs+c2IhRqVQkJiYq/hk9MTFBa2srp06d4sqVK9hsNkpLS3G73RgMhpDFMe3R8vr163IJeWmyOm/ePOx2O6WlpRiNRrRaLZs2bVLcjohEeoDU1tZSW1sb7nBmjUqlYtGiRZw5cwa/3x+WSaBarSY1NZX8/HxGR0fxer2YTCbFDdrToVarWb16NR0dHbS3t1NfXy+feCQnJ4c7vAdKfHx8WNOb5s2bR1paGuvWrcNut2Oz2bhx4wbDw8OkpaXhdrspLS1l6dKlmM3mkA7eMyUiIgK32018fDxWq1UR93nuJN2tW7t2bbhD+UdqtZqlS5eGO4wHVkxMDOnp6Tz77LPYbDYuXLjAlStXsFgsnDhxghs3bsgZCSqVivj4ePLz87Hb7aSnp8+J6qJzQWpqKjU1NRw4cACPx6OIqsTw/xZHfX19+P1+NBoNVVVVuN3uOVeYUKnS09PRaDT09vaybds2JicncTgcVFZWsnbtWsxmc7hDfGBI17gOHz7M2bNnUalUlJSUUFhYGPK7q6p/SE+c8kXpntbdf05ahNz96yz5u79cqfmWYYn5zv8vqb/hNMxYzFI+/n+MYzpC8joPDQ3x3nvv8d1333HmzBnWrFkjpwb8hyJI4v0cGvcdczAYlN/Ld451szjeheR1vvO+zAx8Nh/K90YYPPQx3/lsu9dcZobmIw/96/xXJicnZ3LOd98x9/X18e677/LJJ5/g9/ux2Wxs27YNh8MxW60mHtr3xt1zywfhOTjD7jvmoaEhvvjiC7Zu3crY2Bhut5uvv/56NitU/2XM0z5ZDHU1QuH+KOX/Swl53zNJp9OxceNGnE4n27dvp6WlhfLychYvXjynUkqE6ZEqwD1oHrTPp/BwuPP5poTn3MNGaeNGdHQ0S5YsYfv27WRmZrJhwwbsdvucrWKvZEqZWz6oent7aWtr47PPPqOkpASXy8WyZcvC1spIJO0Lwn8QGRmJw+FAo9EwMjKC0WjEYrHM6fRaQRAEQZiroqKiyM3NZd26daSmplJRUYFOpwt3WIIwbVIRw0WLFsl9hXNzc8O2QTPtNFSFeCiPpMNAxBwaIubQEDGHhog5NETMoSFiDg0Rc2iImEPjgYr5nxaLgiAIgiAIgiAIwkNIWQnngiAIgiAIgiAIgiKIxaIgCIIgCIIgCIIwhVgsCoIgCIIgCIIgCFOIxaIgCIIgCIIgCIIwhVgsCoIgCIIgCIIgCFOIxaIgCIIgCIIgCIIwxf8AVExOP0DDkOUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1152x360 with 20 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "fig, ax = plt.subplots(1, 20, figsize=(16,5))\n",
    "for i in range(20):\n",
    "    ax[i].imshow(x_train[i],cmap='binary')\n",
    "    ax[i].set_title(y_train[i])\n",
    "    ax[i].axis('off')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "#-- Turn the trainign labels (indicating which number each image is), into one-hot encoding\n",
    "#-- e.g. 3 --> [0,0,0,1,0,0,0,0,0,0]\n",
    "onehot_train = keras.utils.to_categorical(y_train, num_classes=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "#-- Make sequential model\n",
    "model = keras.Sequential()\n",
    "\"\"\"Dense implements the operation: output = activation(dot(input, kernel) + bias)\n",
    "https://keras.io/layers/core/\"\"\"\n",
    "#-- first hidden layer has 64 units, and input is 28*28 which is the flattened input data\n",
    "model.add(Dense(64, activation='relu', input_dim=x_train.shape[1]*x_train.shape[2]))\n",
    "model.add(Dense(32, activation='relu'))\n",
    "model.add(Dense(10, activation='softmax'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is categorical classification with 10 classes (0-9), hence 10 units in the last layer. Note that the activation function for the last layer is Softmax. Softmax is a normalized exponential function that returns normalized probabilities using\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " \\sigma(z_i) = \\frac{e^{z_i}}{\\sum_{j}e^{z_j}}\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "where $z_i$ is given by $\\sum_j w_{ji}x_j + b_i$.\n",
    "\n",
    "Thus, the neural network provides probabilities for a given input belonging to each of the 10 classes given by each of the 10 output units.\n",
    "\n",
    "We compile the model with a `categorical_crossentropy` loss function used to train the model.\n",
    "\n",
    "Loss functions are also beyond the scope of this short tutorial, but here `categorical_crossentropy` basically takes in onehot encoded data and calculates the cross entropy loss.\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    " L = -\\sum_{c=0}^{M}y_c\\log p_c \n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "Where $y_c$ is the truth label for categoriy $c$ (either 0 or 1) and $p_c$ is the prediction for category $c$ (probability between 0 and 1). $M$ is the total number of classes, which is 10 here."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(optimizer='adam',     # optimization algorithm used (other examples include scholastic gradient descent, etc)\n",
    "              loss='categorical_crossentropy',\n",
    "              metrics=['accuracy']) # quantity to be minimized"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we can train the model using the `fit` function. Note that we flatten the input images as specified by the input layer when defining our model. `batch_size` refers to the number of images used in each training iteration (every step/iteration that we update the parameters based on the loss function using the optimizer), and `epoch` is total number of times the whole dataset is used. So total number of iterations is `total_size/batch_size * epochs`.\n",
    "\n",
    "We also want to save our model as we train it. One powerfull tool is to create `callbacks` that save the model in after each epoch under specified conditions (for example only saving the best case or all). There are other very useful `callbacks` such as reducing the learning step when we reach a plateau, etc. Here we will just create a model checkpoint to save the progress:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "chk_file = 'mnist_model.h5'\n",
    "#-- create checkpoints that only saves the best trained network so far based on the loss value.\n",
    "model_checkpoint = keras.callbacks.ModelCheckpoint(chk_file, monitor='loss',\n",
    "                                                     verbose=1, \n",
    "                                                     save_best_only=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now train the model and pass the checkpoint as a callback."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n",
      "60000/60000 [==============================] - 65s 1ms/step - loss: 1.9909 - accuracy: 0.8357\n",
      "\n",
      "Epoch 00001: loss improved from inf to 1.99090, saving model to mnist_model.h5\n",
      "Epoch 2/5\n",
      "60000/60000 [==============================] - 52s 862us/step - loss: 0.3734 - accuracy: 0.9114\n",
      "\n",
      "Epoch 00002: loss improved from 1.99090 to 0.37339, saving model to mnist_model.h5\n",
      "Epoch 3/5\n",
      "60000/60000 [==============================] - 27s 457us/step - loss: 0.2731 - accuracy: 0.9310\n",
      "\n",
      "Epoch 00003: loss improved from 0.37339 to 0.27311, saving model to mnist_model.h5\n",
      "Epoch 4/5\n",
      "60000/60000 [==============================] - 29s 482us/step - loss: 0.2230 - accuracy: 0.9412\n",
      "\n",
      "Epoch 00004: loss improved from 0.27311 to 0.22305, saving model to mnist_model.h5\n",
      "Epoch 5/5\n",
      "60000/60000 [==============================] - 31s 512us/step - loss: 0.1956 - accuracy: 0.9489\n",
      "\n",
      "Epoch 00005: loss improved from 0.22305 to 0.19559, saving model to mnist_model.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7f3158010910>"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#-- Train model\n",
    "#-- We will just use 5 epochs to save time, with batch sizes of 32\n",
    "model.fit(x_train.reshape(x_train.shape[0],x_train.shape[1]*x_train.shape[2]),\n",
    "          onehot_train, epochs=5, batch_size=32, callbacks=[model_checkpoint])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2) Evaluate model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 3s 296us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.2150905791155994, 0.9484999775886536]"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#-- turn testing labels to one-hot encoded\n",
    "onehot_test = keras.utils.to_categorical(y_test, num_classes=10)\n",
    "#-- evaluate performance of model\n",
    "#-- Returns the loss value & metrics value, which in this case is accuracy\n",
    "model.evaluate(x=x_test.reshape(x_test.shape[0],x_test.shape[1]*x_test.shape[2]),\n",
    "               y=onehot_test, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The evaluation output has multiple components. The first is always the loss function, and the remaining elements are the metrics specified in the `compile` command, which in this case is just `accuracy`. We see that even with just 5 epochs we get very good accuracy.\n",
    "\n",
    "---\n",
    "\n",
    "Note we could have also just saved the model after it was trained:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save(\"mnist_model_saveAgain.h5\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you have a saved file, you can also load it the weights just as easily:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# model.load_weights(\"model.h5\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "# Regression Exercise with Neural Networks with ATL03-like data\n",
    "---\n",
    "Before breaking into groups, we will do the set up of the problem together.\n",
    "\n",
    "The goal is to now do a regerssion problem. You will have to figure out a way to alter the architecture of the neural network to provide an output that is not a set of classifications, but a regression parameter.\n",
    "\n",
    "First we will create simulated ATL03 data by creating a noisy dataset of a series of patches with linear features and Gaussian noise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#-- create 2000 line segments in tiles with width 40 and add Guassian noise\n",
    "n_tiles = 2000\n",
    "w = 40\n",
    "pts = 30 # number of poits to sample\n",
    "repeat = 3 # number of times to repeat the noise addition at each coordinate\n",
    "\n",
    "# Fixing random state for reproducibility\n",
    "np.random.seed(13)\n",
    "\n",
    "#-- training data has dimensions: # of tilesm, # of points, 2 for the x,y coords of points\n",
    "coords = np.empty((n_tiles,pts*repeat,2),dtype=float)\n",
    "#-- we choose a random slope and intercept in the range -4 to 4 for both, with uniform distribution\n",
    "params = np.random.uniform(-4,4,(n_tiles,2))\n",
    "\n",
    "#-- Now populate x with noisy data\n",
    "#-- to make the data more similar to ATL03, we will have multiple y values for each x value\n",
    "for i in range(n_tiles):\n",
    "    #-- create line \n",
    "    line = params[i,0] + np.arange(w)*params[i,1] + np.random.normal(loc=0,scale=5,size=w)\n",
    "    #-- randomly select `pts` number of points\n",
    "    inds = np.random.randint(0,high=w,size=pts)\n",
    "    for r in range(repeat):\n",
    "        coords[i,r*pts:(r+1)*pts,0] = inds\n",
    "        # Add Gaussian noise with standard deviation 10\n",
    "        coords[i,r*pts:(r+1)*pts,1] = line[inds] + np.random.normal(loc=0,scale=10,size=pts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([], dtype=int64)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#-- Separate 10% of elements for testing\n",
    "#-- randomly extract indices for 10% of elements\n",
    "ii = np.random.randint(0,high=n_tiles,size=int(n_tiles*0.1))\n",
    "x_test = coords[ii]\n",
    "y_test = params[ii]\n",
    "#-- set the rest for training by getting difference between union and intersection\n",
    "jj = np.setdiff1d(np.union1d(np.arange(n_tiles), ii), np.intersect1d(np.arange(n_tiles), ii))\n",
    "x_train = coords[jj]\n",
    "y_train = params[jj]\n",
    "#-- make sure there are no overlapping elements\n",
    "np.intersect1d(ii,jj)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA6kAAAD5CAYAAADFuXPZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOydeZwUxfXAv8UeCKICCygeLGrURI3xwDMxUREPoqJGEF2VgJEIHkRBhR8xMYkkHkDERExQVCKLiBrjhQfirXjgrREEI6t4cYkHCLuw7/dHz7Azs90zPT3d090z7/v59Gdmarpramb61atX9d4rIyIoiqIoiqIoiqIoShRoE3YDFEVRFEVRFEVRFCWJGqmKoiiKoiiKoihKZFAjVVEURVEURVEURYkMaqQqiqIoiqIoiqIokUGNVEVRFEVRFEVRFCUyqJGqKIqiKIqiKIqiRIbKsBuQSpcuXaRnz55hN0NRQuXVV19dISJd873OGHMLcBywTET2TCm/ADgf2AA8JCKXJsrHAGcDG4ELReTRXJ+hMqoo3mW0GKiMKorKqKJEHTcyGikjtWfPnsyfPz/sZihKqBhjGjxeehvwd+BfKXUdDvQD9hKR9caYbony3YGBwB7AtsDjxphdRWRjtg9QGVWUgmQ0cFRGFUVlVFGijhsZVXdfRSkRROQZYFVG8TDgKhFZnzhnWaK8HzBTRNaLyIfAYuCAojVWURRFURRFURxQI1VRSptdgUONMS8ZY542xuyfKN8O+DjlvKWJMkVRFEVRFEUJlUi5+yqK4juVQCfgIGB/YJYxZifA2JwrdhUYY4YCQwF69OgRUDMVRVEURVEUxUJXUhWltFkK/FssXgaagS6J8h1Sztse+NSuAhGZIiK9RKRX166RzEOhKIqiKIqilBBqpCpKafMf4AgAY8yuQDWwArgfGGiMaWuM2RHYBXg5tFYqiqIoiqIoSoJYG6n19dCzJ7RpYz3W14fdIkUJD2PMHcA8YDdjzFJjzNnALcBOxph3gJnAoMSq6rvALOC/wCPAebky+ypKKROmPjHGbGaMedkY86Yx5l1jzB8S5Z2NMXOMMYsSj52K16r4oGMBRVEUf4lCvxrbmNT6ehg6FNautV43NFivAerqwmuXooSFiJzm8NYZDuePA8YF1yJFiQcR0CfrgSNE5FtjTBXwnDHmYeBkYK6IXGWMGQ2MBi4rSotiQgT+O0VRlJIiKv1qbFdSx45t+fGSrF1rlSuKoiiKW8LWJwnvhm8TL6sSh2BtFTUtUT4NOLE4LYoPYf93iqIopUZU+tXYGqkffZRfuaIoiqLYEQV9YoypMMa8ASwD5ojIS8DWIvIZQOKxW/FaFA+i8N8piqKUElHpV2NrpDrthKE7ZCiKoiip5IqtiYI+EZGNIrI3VqbtA4wxe7q91hgz1Bgz3xgzf/ny5cE1MoJE4b9TFEUpJaLSr8bWSB03Dtq3Ty9r394qVxRFURRoia1paACRltiaVEN13Diork6/rro6HH0iIquBp4BjgC+MMd0BEo/LHK4p222idCygKIriL1HpV2NrpNbVwZQpUFsLxliPU6ZoogRFURSlBbexNSLZXweJMaarMaZj4nk74EhgAdZWUYMSpw0C7iteq+KBjgUURVH8JSr9amyz+4L1Y6kiUhRFUZxwE1szdiw0NaW/39RklRdJx3QHphljKrAmj2eJyIPGmHnArMR2Uh8B/YvSmpihYwFFURR/iUK/GtuVVEVRFEXJhZvYmrCTRIjIWyKyj4jsJSJ7isgfE+UrRaS3iOySeFxVyOdEYd87RYkyxphbjDHLEnuLJ8sc9ys2xowxxiw2xiw0xhwdTqsVpTRRI1VRFEUpKVKNsW+/haqq9PczY2uikiQiSNzE5iqKwm1Y8eCpjMbar3gXYG7iNcaY3YGBwB6JayYnvCEURfEBNVIVRVGUkiHTGFu50oqpqalxjq1JTRKxN68DpZd8Jyr73ilKlBGRZ4BMjwWn/Yr7ATNFZL2IfAgsBg4oSkMVpQyIdUyqoiiKoqRiZ4w1NkKHDrBihf01SYP1fxf8lcu/vJjBXR/kyL/+PPR4HD8J26VZUWJM2n7FxpjkfsXbAS+mnLc0UdYKY8xQYChAj1Jy0VCUAFEjVVEURSkZvBpjdcv+Cl9eDP37c2v9UVCV/fy40aOHtbpsV64oiieMTZltXnARmQJMAejVq1cRc4crSnxRd19FURSlZPAUX/rXv8LFloFKfX3rINYSICr73ilKDHHar3gpsEPKedsDnxa5bYpSsqiRqiiKopQMeRtjZWCgQnT2vVOUGOK0X/H9wEBjTFtjzI7ALsDLIbRPUUqSkjJSNb2+oihKeWNnjB18MAwaZL2urIThwxMnl4mBmqSuDpYsgeZm61ENVEVJxxhzBzAP2M0YszSxR/FVQB9jzCKgT+I1IvIuMAv4L/AIcJ6IbAyn5YoSPn7bYSUTk1pfD0OGWAkywIq9GTLEeq6KWFEUpXxI3YR8+HC48caW9zZutF4f8eZETnlhZNkYqIqi5EZETnN4q7fD+eMAdZpXyp5kZv1k4sLkNmfg3Q4rmZXUESNaDNQkjY1WeS50BVYpBew2IU95b5QxRowxXVLKdBNypeSZMqV12UWogaooiqIofhHENmclY6SuXJlfeRLd4FwpIW6j9SbkGGN2wHJR+iilTDchV8qCjRnOdxcxkYmMZBZqoCqKoiiKHwSxzVnJGKle0Q3OlVLBYRNygL8Cl5KeGl83IVfKgoqUqZdUA/WsNukGqnrUKIqiKIo3PGXWz0HJGKk1NfmVJ9ENzpVSxhhzAvCJiLyZ8dZ2wMcprx03IVeUuJFqcG62mVWWaqDWUc+QX6cbqOpRoyiKoijeCGKbs3gZqQ8+CJdeao0iMpg0qbXXVlWVVZ6NICx/RYkCxpj2wFjgd3Zv25TZbjBujBlqjJlvjJm/fPlyP5uoKL6TaXCuWQMjTfoK6jnDqpg8ueUa9ahRFEVRFO8Esc1ZvIzUp5+Ga6+Fiy5qZajW1cGtt6b/OLfemvvHieIG5+p2pvjEzsCOwJvGmCVYG42/ZozZhjw2IReRKSLSS0R6de3aNeAmK0p2cvWPmQbnRUxkvIzkwfb9GdBYz7qN6QYqqEeNoiiKohSK39ucBb4FjTHmGGASUAHcLCJXea7smmtgwwa47jrLSL3uOssiTZC67YBbkuePHWsNSHr0sAzUsLatCSKFs1KeiMjbQLfk64Sh2ktEVhhj7gdmGGMmAtuim5ArMaC+HgYPhqYm63VDg/UaWvrHVMMy1cX3jLX1NDokSerRw6rLrlxRFEVRlOIT6EpqIlvoDcCxwO7AaYmsol4rhIkTrZXU66+39pexcf3Nl6A2OPeyIqpuZ4pXHDYht0U3IVfiyIgRLQZqkqam9K3GkoZlZgzqtrXOWXyj6FGjKIqiKOVM0O6+BwCLReR/ItIIzMTKKuodY2DCBLj4Yvjb3+DCC30xVP3GayIOdTtTvCIip4lIdxGpEpHtRWRqxvs9RWRFyutxIrKziOwmIg8Xv8WKkh9uthobNw4uq0o3UKvbV2U1OIOIpVEURVGUcsLvcMWgjdRgMogaA+PHw6hR8Pe/w/nnF2SoBhED6nVFVBM5KYqieKfui4lc1WTFoJ5BPdvVVrkyOIPyqFFa0HwLiqIopUkyHCd1cW7w4ML6+aCN1JwZRD1nDjXGilG95BKYPBnOO88aXeRJUFsPOK18NjRkV9LqdqYoimJPzq3GJk6EkSOhf3+OW11Po1SpwRkRdJsfRVGU0sVNOE6+BG2k5swgWlDmUGPg6qvhssvgxhs9GapBxYA6rXwak11Jq9uZoiiKPQMGZClPMVBn/LyenrtU6YpdhNB8C4qiKKWLm3CcfAnaSH0F2MUYs6MxphoYCNzv6ycYA3/5C4weDf/4B/+sHE4b00xlJQwfnvtyu4yO2crdYrciakxrr2Q7Ja1uZ4qiKK2ZPdu+vPvMdAP1nOFVsVqxM8bsYIx50hjznjHmXWPMiER5Z2PMHGPMosRjp7Db6hXNt6AoiqLkQ6BGqohsAM4HHgXeA2Ylsor6izEMX/1nxvF//Fr+yT84l+aNzdx4Y25DtaIiv3K32K2IOoXNqpJWFEXJjV1feRETufxLy0Clvp7/+31VHFfsNgAjReQHwEHAeYlM+KOBuSKyCzA38TqWaL4FRVGU0qWNg0XpVO6qTu+XukNEZovIroksooFFVk65yfBbruRKxjKUm/gnv8bQzJQp2a/b6LDphlN5PmSuiNbW2p+nSlpRFCU3mX1lcpuZB9tbBipVVbFcsRORz0TktcTzb7AmdbfDyoY/LXHaNODEcFpYOJpvQVEUpXRJjbY0NNuW50vgRmqQpGYKtIxKw+X8iT9yOedwM1MYSvPG7L+Ok+HoVF4IqqQVRVG8k9qHJg3Ueyr68/Vky0CF+K/YGWN6AvsALwFbi8hnYBmyQDeHa7wlICwimm9BURSldNl8c+txIHfwEgfSgW/Syr0QWyM1M1NgC4bf8wf+wO/4FVOZan6V1YwvpuGoSlpRFMXCy3YkyT70j51aVlDXT63n9EFVm86J82SgMaYDcA/wGxH52u11BSUgLCKab0FRFKU0Wb9mA9cyijs4ne9ox2asA1onzMuHSp/aVnTsMgW2YLiCPyAYrpA/wNkCN99sG2iaVJJjx1ruYD16WIOZoJRnXZ0qZkVRypvkJGOyD08mN4L0/rG+3qZv/mIiJGJQj6tvWUFNUuw+3S+MMVVYBmq9iPw7UfyFMaa7iHxmjOkOLAuvhYqiKIpiw4oVPMxAjmQuf+N8LmYiG7B0s1M+HjfEdiU1V3xRRQUsG3YFXHEF3HYbDBniGGiqs7uKoijFw812JHb7ar492Mrie5fpT9Vd9VS2q7JNjhe3Pt0YY4CpwHsiMjHlrfuBQYnng4D7it02RVEURXHkjTegVy9+wnP8klu5kL9tMlALJbZGqlN8UTKL7oYNMHky8Pvfwx//CP/6Fwwe7E9GJEVRFMUzbpIbZRqyFzGRq5pGMov+nC71bKCKjRtxlcU9BvwYOBM4whjzRuLoC1wF9DHGLAL6JF4riqIoSqC4CsmZMQMOOQQ2bOCozZ5lGr9sdUpZxqTmFXd0+eXwpz/B7beroaooihIynTvnLk81WJNJkmbRnzrqW83S5sriHnVE5DkRMSKyl4jsnThmi8hKEektIrskHleF3VZFURSltLHzZErbb3zDBrj4YstNqVcvePVVfn3z/q2iKisq4J//9N6O2BqpeSch+u1v4corLUN10KBIG6peEoooiqJEmdR+beVK+3PWrWt5nvSWyWWgQqS7c0VRFEWJFVlDclasgKOPhr/+FS64AObOha23pq4Opk1Lt8umTSss3Ca2iZPAQxKisWOtX27sWGtqYNo0qIzWT+A2oYiiKEpcyOzXnFizpuX5uHHwzuAJ/KVpVFYDFWxz4ik22CaiUr2iKIqipOAUklPT8BrsdxJ88YWV72fQoLT3/U4OG9uVVDtcrUD+3//BX/5i+VGfdZa1ZB0h3CQUURRFKRZ+eHZkz8ZuT93nloH6YPv+nEE929VW0bu3/bnJiTzFmZzuW4qiZMUYc5Ex5l1jzDvGmDuMMZsZYzobY+YYYxYlHjuF3U5FKRS7vD91TOd582MrI+Fzz7UyUIOgZIzUvBTw6NFw1VVwxx1w5pmRMlTdJBRRFEUpBn4ZNm77r5qaxJMJE2DUKGubmdX1NEoVS5bA44/TylDt3TuRJE/Jik6AKop3jDHbARcCvURkT6ACGAiMBuaKyC7A3MRrRYk1ffu2PK9gAxO5iOmcyUfdD4RXX7XiUItAyRipeSvgyy6Dq6+GmTPhjDMiY6g6ZS12KleUJMaYW4wxy4wx76SUXWuMWWCMecsYc68xpmPKe2OMMYuNMQuNMUeH02olyvhl2Ljpv6qrYdIk0gxUMvZBra+HefPSr5s3T1cD3aAToIpSMJVAO2NMJdAe+BToB0xLvD8NODGktimKb8yebT12YTlz6MNFXMckLqRv5Rzo1q1o7SgZI9WTAr70Urj2WrjzTsuJOgKGal5ZixUlnduAYzLK5gB7ishewPvAGABjzO5Ys8B7JK6ZbIzRyD4lDb8MG7t+rarKWjlNJli45RbLxdfJQAVdDSwEnQBVFO+IyCfAeOAj4DPgKxF5DNhaRD5LnPMZULwRvKL4RGZYT0MD7MNrzKcXBzOPs5jGb5jE/z7Ovv+p34lfS8ZI9ayAR42C8eNh1iw4/XRoavK9bflgl7V40CBrEKbZfpVsiMgzwKqMssdEJDn78iKwfeJ5P2CmiKwXkQ+BxcABRWusEgv8Mmzs+rVbb7WSBDY3w5IluQ1U0NXAQtAJUEXxTiLWtB+wI7AtsLkx5ow8rh9qjJlvjJm/fPnyoJqpKHljF9ZzJrfzPD/GIPyE57ids4Dsuj+IvAclY6S6VcC2Vv7IkZaL2V13wWmnRcJQXbLEGryNG2clIdZkF4oPDAEeTjzfDvg45b2liTJF2YSfhk1qv7ZkSUYGQAcX38z+2ml/VV0NzE3e27ZFBN2STYkIRwIfishyEWkC/g0cAnxhjOkOkHhcZnexiEwRkV4i0qtr165Fa7Si5CLVQ6mSJv7Kb/gXZ/EiB7Efr/IqVvxpLt0fhKdTyRipbhRwfT0MGZJu8A0ZklB6F19s7flzzz2RMFSTqHub4gfGmLHABiA5xDM2p4nDtToDXKYUxbDJYqBmzsp+/bUVu5qKrga6J+tEQR4Uy3DUjMRKhPgIOMgY094YY4DewHvA/UAyzekg4L6Q2qconkh6InVlGY9xFL9hEtcxgqN4jM1ru7rW/YF4OolIZI799ttPgqSmRsRSdelHTU3KSX/9q1V40kki69cH2h43GGPfZmPCbpkSFMB88ShDQE/gnYyyQcA8oH1K2RhgTMrrR4GDc9UftIwq5cP06SJ/6jReBOSB9v2l/rbGtPdra53769paqw+srbXqKTaFyGjQR9AyOn26SPv26f9J+/bB/A9O90Btrf+fpZQWQcgo8AdgAfAOcDvQFqjByuq7KPHYOVc9qkcVr0yf7r/+q60V2Zf50sAOspbN5Az+5amfzbe/diOjJbOSCrlnd1eutL8urfw3v7FSTN57L5x6KjQ2BtRad2iyC6UQjDHHAJcBJ4hI6pr8/cBAY0xbY8yOwC7Ay2G0USkPUvvnLl3grUET+O2Xo5hFf05aW885w6vS+myn2ddVq/xZDVS8rYgW07tHY5CVKCEivxeR74vIniJyplg5HVaKSG8R2SXxuCp3TUq5M3w4VFZaHkqVldbrXP1xfT0MHpzuWTJ4cOGeJTOO+RfP82MAfszzTOdMTx5KqdvWuCl3RS4rtphHIbNLbmZ37Sz85NGK66+33ujXL9QV1WLOWivRAI8zwMAdWFkHm7BiTM/GSoj0MfBG4vhHyvljgQ+AhcCxbj5DZ4AVL2T2YxdjraDeSX+ppNF2xtWV50tIeJXRYhxuZdSrbimmd4+upCpeKQUZVUqTYcPs+7U2bbL3x77rxMZGkQsvFAH5bPfDZZ/tlxW0QqsrqVlwM7u7aaP4DGzLL7gA/v53uO8+K1YqpBXVuCa7UIqPiJwmIt1FpEpEtheRqSLyPRHZQUT2Thznppw/TkR2FpHdROThbHUrSiGk9s8XM4EJWCuoddSzgZYsvrpCVjy8rogW07snkJl5RVGUEJkyxb68uTn9dWZ/7Mob1C3LlkGfPnD99XDRRTxx2WOsqigsoVcQni8lY6S6+XEmTWq9q0FVVWIDeTvOOw9uuAHuvx9OOQXWr/elrfniV7ILRVGUMEj2wxcx0dFAhXRDZ5WD05xTuZIfXgcUxdzKJrmhvNtyRVGUqLNxo/tz/Zi4zXQjfvhP86FXL3jpJbj9dur3m8g5wyoLTlAXxARmyRipbn6cujprb77MvfqyGn3Dh8PkyfDAA3kbqpo6X1EUxeqHL2IiExnpaKBmGjoajx8sXn/fYnr3NDTkV64oihIkfozrKyrcn5vaHzt5g7Zpkz2ONTVD+s8apnH4737Ct9+1gRdegDPO8C3PQCATmLn8gYt5BB2TWhD/+IdV6c9/LrJuXfjtUUoWNJZGKTHmnz5BkjGoFTQJiFRVWbE0TjEwUe5DS0FGo/z7JqmokFbxTWCVK0o2SkFGlWjhV5/pNiY1U0cOGyZSXW1/rVN7knGilTTKJC4QAXmcI2Tv7ZdvOsfPPAP5ZB92I6OhC2vqUajgBpGaOY1//tP6yfr2Ffnuu6ynasIHxSuqXJWSYoJloC45oL/s1KMpr/458D7dI6Uio1H9fZPklexQUVIoFRlVooOf4/revdPr6N07vT+uqWltkLZvbxmqyXOcJvFS22OMSDc+l6f4qQjIeC6WCprSDNCwtntzI6Ml4+4LRYjdHDrU8muaPRtOPhnWrXM8VVPnK4pSChTk3jRxIowcCf37U/v8DD5oqMyrf9Z4/GDx+vsWK5Sltja/ckVRlKDwa1xfXw/z5qWXJV8n++MOHVrna1271jI/kudkJlqya89xW7/CfHqxP69wOvWMYgIbqUxzI7Zz062qgm++oeA41UIpKSO1KJxzDtx0Ezz8MJx0kqOh6hTX07lzunIfPry1stdYVkVRokBmPEteiirFQGXGDOrvrMzZ9ynRp6B7Ik+KmaRJaY2ORRSlBb/yJDjFgI4Y0SJvTnH3qQZo587257Rvb9Uz2NzGrM8PZSMVHMIL3MHpm85JzZBul2dgyy3tjeQg9sPOSq6l1mIesXKBmDrVWgM/6iiRtWtbve3W5zzzqK62/NCjHCekBAvqpqREBM/uTeOtfVClf3+RpibbWB67vi8u/Vwpy2guF+Bih7JE3SW5VIlDzHI2SllGlXDwSyacYkDdHKl7ojrtm1pJo/yN80RA5tBbalied39djFALNzKqK6leGTIEpk6FOXOgXz/47ru0t51S5DstzydpbISmpvSyUGYvFEUpezy5N02YAKNGbVpBpbLSduY4k8ZGaya5XDHG3GKMWWaMeSelrLMxZo4xZlHisVOQbXCzSlrsUBZ1+Q4HvzJ+Kkqp4Fdmc78y1Nttx9aNL5hLb87nBsYzkmN4hJV0aXVerv7aKQNxPpmJ/UCN1EIYPBhuuQUefxxOOCGtR/dbYWssq6IoxSZv9yYbAxXc91+eNiUvHW4DjskoGw3MFZFdgLmJ14HhxjDRrYHKAyeZbWhQF2ClfPFj0swujMEtqYZpZp+7Py/zKvvRi/mcxgwuYTwbqbStJ1d/7bSXaz57vPqBGqmF8stfWputzp2bZqg6+Yp7RQcAiqIUm9S4lZzlDgYq+N8fliIi8gyQOTfeD5iWeD4NODHINrhZJdU40fLAacxhTPjJVBQlztityHbo4O5ap4RHg7mFZzmUJqo4hBeYyWmOdbjpr6OStE6NVD8YNAhuuw2eeAKOPz63X1ue6ABAUZQwcApbaFWexUDNB6fNysuYrUXkM4DEYze7k4wxQ40x840x85cvX+75w9yskvrl8lZuxC0Jkd1khDGWcZqKugArSv5krsi2bZv7mkxboK4Obp7cyLQO53ELZ/PyZj+lF/N5k71tr8+nv47KZKQaqX5x1lkwbRo8+SQcdxzrVq5xPDWp3LOhAwBFUcLGVfyhCwPVLnYmk6oqmDTJWzvLHRGZIiK9RKRX165dPdfjdmASdpxopsEX9UzRxcyI7Bd2kxGZBmoSDUdSlMLIpiMdbYHPP+e0m47grG8nw6hRHPrNw2xRaz/TW1ubX38dmcnIXJmVinmURMaz228XadNGXmh7mLTn26wZtQrJkqgZD0sXNCuhUiQKzuSakcXXCad6Kiri2YcFJaNAT+CdlNcLge6J592BhbnqCDq7b9i4yRQdtSy0xc6IHBRx+h6qR5U4kbdsvfiiyLbbirRrJ3LHHZuKvWYgDqPfdyOjupLqN2ecAbffzoGNz/Bwm5/TnpYV1cwZaa/L6XGclVUUpbjkci90049kjUnNw8XXqa+bNk2ztubgfmBQ4vkg4L6gP9DNKmmYrqtuMkVHzQW12BmRgyIqLoCKUmrklf9h6lT46U8tH+F582DgwE1v1dVZEYjJLLwVFdbrbPo10jZFLiu2mEdJzS7V18tG00ZebPtT6cA3jjMTXmYv4jSbqeQPOgOsFIib2VQ3/YjTPmyXt3e3gprKsGHWymlyBXXYMF+/clEJQkaBO4DPgCZgKXA2UIOV1XdR4rFzrnqCltGw9890u8egMcVpjxtKSWdHfaU9iepRJU646iPWrxc591zrjT59RFasaFWPXf9cVWXp8nw9pmpqgpV1NzJaqFK9FlgAvAXcC3RMeW8MsBjLXeloN/WVnODecYdImzYihx4q8s03vlXrpKSjpJQV73hVrsAtwDLS3QU7A3MSg9w5QCdRGS153Cg8N/2I3fsXk7+BOn26pSgzFWdUB7i5KOcBcNgGl9PnR9kADNuwL0fKWUYVb4Q5AZNTH3/6qcghh1iFl17qqHvd9I+ZfY/biT+/+yw3Mlqou+8cYE8R2Qt4PzHoxRizOzAQ2ANr37fJxpgibwEbAQYOtNzgXnjBWrP/9ltfqtV96hQHbsPlPosqo6WNG/dCL/3IxUxgAqOYRX5ZfEeMgKam9LKmJqtciRdhu6662WMwai6okUlCoiiKLWG7vGbVxy++CL16wRtvwMyZcPXVjrrXTT/sdu/rXNcVg4KMVBF5TEQ2JF6+CGyfeN4PmCki60XkQ6zVmgMK+Sw7YpHS/dRTWwzVY4+Fb74puEqNC1HskPz2WSyKjCrh4LQvaWq5m34kdUuYVAP1gs75bTOzcmV+5Up0CXuS1M7gGzYs+gZg2BmRFUVxxi7WvZhG2bhxUF2dXlZdDTOPvBl+9rOW+NNTT81aj9t+ONfe126uKwZ+Jk4aAjyceL4d8HHKe0sTZb4R9qxHXgwYAHfcYd1gPhiqfs7KxsLQVwrBaZ/FwGVUiTZu+pFJkyxFmWqg/rJqBhOv97YPqhJ/ojBJmmnwTZ6sBqCiKN4J20MELFsmSRWNXL9hGAdNPccyUufPh732ylmHW4Mz197XTnuWF91jM5c/MPA48I7N0S/lnLFYMakm8foG4IyU9zUg5h4AACAASURBVKcCv3CofygwH5jfo0cP177MYcfFeOKuu6yMIYccIvLVV2G3RuNkIgoFxNLQeguL1RnvfylFklElPPyMW59/uhWDeif9ZaceTZ76B6cETDU1+dcVBQqR0aCPYmxBE5fkOUo4ROH+KGUZVfwnbJsi9fO34VN5Div+dPKWl4ls2JBXXanyV1MjUl2d/zi/GPaBGxktWNiw0uLPA9qnlI0BxqS8fhQ4OFdd+QhubJMH3X23SGWlyMEHi3z1VaidedhCqdjjs5Fqu89iMWRUCQ/fZDt1H9TGRs/tmT69taKsro6vcVOqA2CduFQKJSr3UKnKqBIMYd+3SZvmIF6QT+gu39Je+nOnLzaNVzsjaPskcCMVK+HKf4GuGeV7AG8CbYEdgf8BFbnqy0dwY21g3XOPSGWlLPveQbJNu9WhC0XsDP0Sx2cj9VpgdOL5aOAaKZKMKuHhi8L1yUBNbVPYqyt+UaoD4FjrVSUSROUeKlUZVYIj7EWjXzFF1lMli9hZ9uQtV3ITZ73qRkYLjUn9O7AFMMcY84Yx5h8AIvIuMCthwD4CnCciGwv8rDSiEBfjmZNPhlmz6Lh4Pvd+dzRb8tWmt4oZqB12AgzFX4wxd2B5NexmjFlqjDkbuAroY4xZBPRJvC6KjCrhUXDc+oQJMGoU9O9vBapXVfnSJo0bjDZRiMsqVcol/0Mp30PGmI7GmLuNMQuMMe8ZYw42xnQ2xswxxixKPHYKu52KN0LTUevXc/vmv+YmhvIER3AAL/MOPwSsjUGciFVuHq/ksmKLeeQ7uxTnGQQRkX78R9ZTJS9ygGzFl0VfyQzbvUGxB50BVsLE5xXUUqRUZdTPVTA7/Rx3ne2VQnRt3H6zUl5JxcqQ/6vE82qgI3AN6d5KV+eqR/WosolPPrHC/0DGMUbasMG13ERF1rziRkZDV6ipR6kJbi7lUlsrcjz3yXqq5CX232SoFvMGi5sCLAdKdQCsxAA1UF1RqjI6fbpIVVX6gKeqKn+9YGeUVVV5S+ARRzL1qlPiMDeufHGbSI5Km/2WUWBL4EMSCUJTym3zPmQ7VI8qIiLywgsi22wjsvnmcgp35R1+F/eQPTcy6ucWNEoKbpbhx42Due1P4Bfcw968wRz6sG27L0NN5a8ueIpSpqS6+M6Y4YuLrxI/jMn+2g12ew42NUFjY3pZGJvDB42d7nfaDziXC2zYezd6wc8t8iLGTsBy4FZjzOvGmJuNMZvjvM2bojjzz39aW8tsvjm8+CKv1J5ie1q28LtyCNkrOyO1WHEhbpRLsjN/u/Z4fsG/2Yu3eGubPtT1/TKYRimKotiRaaBW6j6o5cjYsa0NycbG/I2ifOIPc50bt1hOO93vRK7BZFzjO0t08rsS2Be4UUT2AdZgufe6whgz1Bgz3xgzf/ny5UG1UYk669dbs1jnngu9e8Mrr8Cee3rKsxPr3DwuKSsjtZhBxm6VS7Izf0COo+2D/6bmk7fhyCNh1Sr/G6UoipJCfT1c2dkyUB9s358Zx7U2UONmJCg5+Oc/4aabYMECSxGm4JdRlM9MfrZz45gYxO1v5WYwWQ4rJTFiKbBURF5KvL4by2j9whjTHSDxuMzuYhGZIiK9RKRX165di9JgxX8K0oeffgqHHWb1v2PGwIMPQicrz5YXD4QS9lpoIZc/cDGPoP30ixlk7PmzHnrICtzZZx+RlSv9b5gSeSjReDclWkyfLjK6yopBvZP+UkFTq9ixqMSXRY1Yy2ivXi1/ZpcuIieeKDJhgsjLL8tOPZp80ZF+xaTGMTGIU5travLP/6Dy550gZBR4Ftgt8fwKrC3ebLd5y3aoHo0nBcnjc89tij+Vu+4KvK1xwI2Mhq5QU4+gBbeYQcYF3cyzZ4u0bSuy994iK1b43zgl0sR6AKxEhlxJ0f7UKd1AtTMA4mgkFINYy2hzs8iCBSI33SQyaJDITjtt+mMb224uT7TpLb/jCjmcudKebz0bRX5k941jYhC/DUtNbuiNgIzUvYH5wFvAf4BOQA0wF1iUeOycqx7Vo8XHDznypA+bm0VuvNGapdt5Z5G33/bU/lJEjdQMij3gKkgoHn5YDdUyJdYDYCUS5Bwoj7c3UDMNgDgaCcWgVGQ0qaO25RMZ3uVOWdDnfFnZ40eyEeuPb6RSlu98gMjIkSL/+c8mXVQswymukyRqWIZPqcioUjheJ46GDROpqLDOTz7mpQ/XrRM5+2zrpGOPFVm1KpDvFte+Ro3UDGLnOvPII5ah+qMfiSxfHnZrlCKhylUplKyD+4SB+kD71gaqrqS6oxRkNKs+/PJLK/RkzBiRn/wkzU/3y+12l5srh0odt0sPlgSqR2Ons5XIUAoyqviDFz02bJj9Na7rWbpU5MADrRPGjhXZsMH37zV9eusQiurq+PSPbmS0rBInxS7I+Oij4f77YeFCKwvYihVht0hRlBjglLzllIaWLL5f3ziDtu3TkyRlJnMph+yB5UrWDPQdO0LfvvDnP8Ozz8JXX8Ezz8Cf/8zrK2s5ZcNMpnMmDfSkgR7ctPZ03rvwRnjnHSulq0/ETmcrihI5vCSEmzLFXd22+vC552C//eDdd+Gee+DKK6Giwl2FeTBihH029hEjfP+o0CgrIxVimBr9qKMsQ/X99+GII0BTlyuKkgO77J8XM4HxtGwzc/pZlTkNADUSSpe8Bm6bbQaHHgpjxtB7/Ww6s4q9eZ3z+RvzOJjDeIorVw2HH/4QunaFE06Aa66BefNaj6LyxI3O1gzUiqI44SVL9saNzu856kMRmDwZDj8cttwSXnwRTj7Zc7tz4bT/slN5HCk7IzWW9OkDDzwAixZZK6oxMVR14KAo4ZC5AnoxE5jAKBoOyH8f1NhN7Cmu8Lq9SY8e0EwFb7I3N3A+A7mT7fiEQ7f9AG67DU46yZpUvewyOOQQa1X28MPhd7+Dxx6Db77x9XvEcZsaRVGKhxePIKeFz4oKB324bh386ldw3nnW4tLLL8Mee/jQ+vJGjdS4cOSR1p5KixfDEUdw9+RlkTYAdeCgKOGRugI6MsVArX2+xUBVGY0HxphjjDELjTGLjTGj/arXqyu3/XWGc6/ZCQYNgptvtvZg/eILy9Xt17+2DNNx46wQlk6doFcv+M1vrPe/+KKg75HVbVlRlLLHi0fQ0KF5lH/yCfzsZ3DLLfDb31qLSh07FtzuXAs9NTX21zmVx5JcQavFPMohmNxLJq7UawZ2myvrK9vJu2YP6coXkU0moQlXvIMmfFD8IpEkSfr3F2lqSntLZdQ7xZJRoAL4ANgJqAbeBHbPdo2X7L75Zob0dN3XX4s89pjI5ZeLHHaYSLt2LTfdLruIDBkicsstIosWWds2uCRbQhOlfFE9qhRKZnbfYcNsTnrmGZGttxbp0EHknnsK+rzUfrWmxtq1JjNp3LBh6ee0aZN+TlVVtGyBbLiR0dCFNfUodcH1kqnQ7prDeULW0E7eYXfpxueRHFzq1hXeUeWq+EIWA1VEZbQQimikHgw8mvJ6DDAm2zWxkdH160XmzRO55hqR448X6dSp5Sbs3t26b6+/XuS117JmxnTaGqKioojfRYkcqkdLk8hsudLcLHLDDSKVldYk27vvFlSd3VjfST9nZvOtqYnA7+EBNzKq7r5FxItbkt01T3I4fZlNT5bwJIezNZ8D2TOVFRuv8U6KovjAhJYsvk4xqCqjsWA74OOU10sTZaHiS76B6mo46CC45BIrOeCKFVZ24BtvtGJYX3wRLrwQ9t0XOneGY49tyTa8bt2mapwSnGRLfKIoSvyITIjKunVw9tlW/Okxx1jxp7vvXlCVdmN9O6y5yhYaG6FDh9LNGaFGahHxkgbb6b2nOYy+zKaWhk2GapQGl7p1RbQwxlxkjHnXGPOOMeYOY8xmxpjOxpg5xphFicdOYbdT8QEXBiqojMYEY1MmrU4yZqgxZr4xZv7yPBLr2RmbuQzQwAaKbdpYiUbOPdeq7KOPrFHX9Olw2mnW67Fj4ac/ha22gp/8BMaMYVC32WzF6lbV1dYW2B5FUSJFJOLPP/7Y6oNuvdVKBnfffb7EnxayyBSlBSrfybXUWsyj1F0gvMSAOV2TXPI/lKflGzaX98z35e6/fVqkb+KOyLhlxAx8dlPCWnn5EGiXeD0L+CVwDTA6UTYauDpXXaUuo7Enh4tvJq5ibpRW+C2jTgc+u/vminmqrraPg0rtu0ONZV6+XOS+++TdvqPkteoDpZFKEZCNGHmDveRvnCcDmCk7bfaJ6psyp1gy6uVQPeqN0ENUnn5apFs3kS22ELn3Xl+rdupXs7n6poY2xHGc7UZGdSW1iHhZuXC65txzrZni58xPGbz1w+xc/TG/+Pvh8Nln/jfcBjfuXrp1RaSoBNoZYyqB9sCnQD9gWuL9acCJIbVN8QOXK6hJ6uth2rQWt8iNG63Xmt03UrwC7GKM2dEYUw0MBO73UlHmCujKldDUlH5OY2PrssyVCrceQYFsQdalC/XfnMD+T13Lvo0v0pHVHMFc/lRxBV9WdeOX3MadDOSDddtRd3lKtuGFC1v7ySmKEitCC1ERgb//3doCslMneOklONHf4dK4cVBVlV5WUWFl6k1mJD733Nb2AFi6W8J0fw6SXFZsMY9ymF0qNLuv4zXPPmtlF9t1V5FPPvG1zXbtyTcBlOIeApgBBkYA3wLLgfpE2eqMc77MVU85yGiYePY+yHMFVUSz+xZCEDLqdAB9gfexsvyOzXW+k4y6manPNoOfq57U+yZIHZH18xsbRV5+WWTiRJGTThLp0qXlhK5drbKJE0VeecW1nPiFehYVl2LKaL6H6lFvhDL2/O47kUGDrA87/niR1asD+Zjp0y1PlkzPlszvltqPOCWNi4sOdyOjoQtr6qGCWyDPPVcUQ1UHt8Hit3IFOgFPAF2BKuA/wBlujVRgKDAfmN+jR48i/QrxJ99BqWcF7MFAFYmA61SMieMA2On/dnPka4AGqSPyum+bm0Xee0/kpptEzjpLZKedWi7YfHORI48U+cMfRJ54QmTNmsIb54BO7BafOMqokpuiTvZ89JHIfvtZAnvFFSIbNwb2UV76zLjrcDVSy5GkobrLLiJLlwbyEXEXjKgTgJHaH5ia8vosYDKwEOieKOsOLMxVl8qoO7wMSj0N7D0aqJ4/TxER/2XUz6OQlVQ3MakiuQeKfuqIzM+qqSnwvl26VGTmTJHzzhP50Y9aGltVJXLQQSKjRoncd5/IihX5N9YBlbXiE0cZVSLEU09Z3hdbbGH1BwHjpc+Me7/iRkY1JrXU+PGP4dFH4fPP4bDDYOlS3z9Ct66IHR8BBxlj2htjDNAbeA8rtm1Q4pxBwH0hta/k8JKFMO/s33nGoGbSt29+5Uq8sctvUF2dHvN0yy1W0sra2payKVNa5xPIlW/ASRd07pxfnKpdJuFvvmkdu5VXVurttoNTT7VizN54A1atgocegpEjrSCw66+Hfv2gS5fW2YY94iWzv6IoISBi9QG9e/NVZWd6b/EybU48wb+4ege8jKvLIkN/Liu2mIfOLvnICy9YM0A77yzy8ce+Vq2uS8FCMDGpfwAWAO8AtwNtgRpgLrAo8dg5Vz0qo+4IfFa0gBVUT5+npBGEjPp1uM3uG6SrnJ2OcLtKm4rTPVpTE+D3+O47kWeeERk3TuSYY0S23LLlg3fYQeT000VuvFHk7bddu/+prBWfuMqoEiJr126KP/1o3xNkm3arizbO9TqujnOsuxsZDV1YUw8VXJ+ZN89SsDvvbPnW+0icBSPqqHKNP14Gpa6VlA8Gqoi67RdCucuom/7fDzfdbHG0RdM/GzaIvP66yPXXWzK3zTYtjejc2Uqmcs011sTw+vW2333YMJ3YLTblLqNKnjQ0pMWf9uyxsegTS+U2rnYjo+ruW8ocdBA89hgsX265/vroW1Sq28sEsm2CUnZ4ccOpq7NcK7O6Whbo4puKuu0rXqivhyFD0l1whwxp3Vdm6ohVq+zry6aWnO5FY9I/P9BtFyoqYO+94YILYNYs+PRTWLzY8os+8URYsAAuvRQOOQQ6duTz3Q9nyVm/Y9eGx9hcvqGhAaZOhYMPtqpKVjloUOnozUxUjyrFpOD77amnoFcvWLQI7rsPfv97Gj62N4/URb/I5LJii3no7FJAvPSSyFZbiey4ozVbpNgSFTdmdAa4JPB9VtSnFdTU9kXhfo8j5SyjTiuiNTXZr/PLu8BpdTVU19nPPxe5+26RESPkjYp9ZQNtRECaqJBX2E8m8hs5iXukK1+UvKxFpV8pZxktJwq635qbRa67ztrL5fvfF1mwYNNbxXbRj4rcFBM3Mhq6sKYeKrgB8vLLLYbqkiVhtyaSRCVuSJWr0gqfDdQk5eZe5BflLKNO7reQ/Tq/Yq6cPjsqbuogsgVfSR8elT/yW3mSn8laNtvU0AXsKjdxtpzFbfKTbT+wBsoxIlefoXo0fBktJzzfb2vXipx5pnVyv34iX32V9naxjcaoyE0xUSM1ZCI3AHzlFZGOHUV69hT58MOif3zkfo8MohKjp8o1fgR6bycN1AEDfDVQFe+Us4x6NVJF/JGTqA/m7NpWzTo5mOflUq6S+zlOVtGx5c1tt7Vk+29/E3njDSsGNqK4GbirHg1fRssJT/dbQ4PIvvtaJ/7hD44J0Io5Zo2K3BQTNVJDJOhZGM/CM3++ZajW1hbVUI2DK0NUBj+qXONFoPe2GqiRpJxl1Ku7r19EXZc4/T5pA082yp68JcO4QWTgQJHtt295c6utRI49VuTPfxZ59lmRdevC/kqbcKMjVY+GL6PlRN7325NPinTpYiUVfeCBtLfCXEiJitwUEzVSQyTIG65gJf3qqyKdOon06CHyv//Z1u+3oMZBAKMy+FHlGi8Cu7fVQI0s5Syj06e33kqmqqq4/WSUvXLsfp+cq8/NzVYYzu23iwwdKrL77i0ntG0r8pOfiIwZIzJ7tsjq1aF9NzerPapHw5fRcsL1/ZYl/jRZT3V1ej3V1a3rCarviYrcFBM1UkMkyKV7XwbFqYbqBx9sKnba366mpjChjIsrQxQGP6pc40Ug22QUyUCNwv0eR8pdRvW+yU7m79OmjX3/UFGRpZLly0X+8x+RUaNEDjxQpLKyRWn+6Eci558vcuedIp9+WqRv5X7sEYX7o9xltJzIeb+tXStyxhnWzWoTfyrizkMksh6SMUWN1BAJcuXQN4Pvtdesfd522EFk8eKs7S501jwOK6lRQZVrvHC6tzPl1LUyK6KBWm4zt35RyjJabgOlYjBsmH0fMWxYHpV8+63I3LkiV1wh0ru3yOabt1S0004igwaJ3HSTtUIUUDKmOPUZQckoUAG8DjyYeN0ZmAMsSjx2ylWH6tEismSJyD77SLMxMn6rP0obNtr2azm9HUTHsX6jRmqIBNmZ+ykoD417XVa16Swfsb38dNtFOQ1Uu9klN8RJuYVNKQ+ASxE/t8m4+2DLQJ3JAKlu05TfIDZPVOF6p1RlVPvp4Bg2zFo5Ta6gFizbjY1W1v4JE0ROPNGKs0v+ad26iZx8ssjEiVbCxDLMCB6gkXoxMCPFSL0GGJ14Phq4OlcdqkeLxNy5IjU1sr7dlnJy2wey9mtujNS4eATGBTVSQybqvuvJevbiDVlOjXzMdvI93BuqXtodB+UWNqU6AC5l/NgmI9VAraDJ22pLHqjC9U6pymgUJi5UT3ikuVnkvfdEpkyxttbo2bPlD+zQQaRPH5E//lHkiSdE1qyxrcKv3z4K/2EQMgpsD8wFjkgxUhcC3RPPuwMLc9WjejRgmputCZqKCpHdd5fDtl2Ys19z4+4bhf6xlCiakQqMAgToklI2BlicEOCj3dSjgusev1P5/5A3ZRld5GO2k114PxAjVXFHqQ6Ay4m8ldl4ewM1ueoSiTYqmyhVGQ174qJUVnKjYKSJiMjHH4vccYfI8OEiP/xhyx9cVSVy0EEil1wicv/9IitX+j75HfZ/GJCRejewH3BYipG6OuOcL3PVo3o0QNasEamrs268k04S+fpr1wm/ciWEi8q9XSoUxUgFdgAeBRqSRiqwO/Am0BbYEfgAqMhVlwpucckU3D15S5bRRZayrRy27UIxxjnhQ7G2GxCJkMIvEqU6AC4n8lJmWQzUICeEVOF6p1Rl1O32MkH1yaUwcRJpuVq1SuTBB0Uuu0zkkEPSRuULqvaQyZwrp1EvO9Dg+bd3+g8rKoqrw/2WUeA4YHLied5GKjAUmA/M79GjR/A/QDny4Ycie+9t3Wjjxm2KzfYz4Ve5jUeDpFhG6t3Aj4AlKUbqGGBMyjmPAgfnqksHwPYUc0CwB2/LMtNVPq/oLruyMC03Q+rhxQXRSwcwbFiEFX5AlOoAuNxwJbfXXmvd1AOsGFSnwV2obVRaUaoyGnaGy7BXcv0gVob22rUiTz8tcuWV8jBHy1dssanBS+ght1MnQ/mH7ME70rPHRlf/cbZs58XU4QEYqX8BlibGup8Da4Hp6u4bER5/3OqottpK5KGH0t6K9MRRGRO4kQqcAExKPE81Uv8OnJFy3lTglFz1qeC2JkjhctpuZu/Kt+ULusondJddWeCLwnXzPfxMQBNnSnUAXEr4kgQlxUCVpiZ/MoAqRaFUZdSNkRikERYrA8+BuBratbUiFTTJPrwqFzBJZnGKfMbWm77AcmrkgYoT5NXTrhV58UUrcZNDPbmM1GL8p0HKaMZK6rUZiZOuyXW96lEfaW62koe1aWPtL/z++7an6YRs9PDFSAUeB96xOfoBLwFbJc5LNVJvsDFSf+FQv7pAZCFopZ0puMmZ9N15Rz6nm3xCd9mN9wpWuG6+h1vlFgeFXwilOgAuFXwxJjMM1NS6fc0AqgRCqcqom346SCOsFCZq4mpo200SQ7N8j/dlMFNlKoNlIbu0vNmuncjhh4v87ncic+aIfPNNlnqKr8OLaKTWYCVTWpR47JzretWjPrFmjcjpp1s31C9+IfL112G3SMmDQFdSgR8CyxLG6RJgA/ARsI26+/pHsWdlUz/vB7wrn7G1fMo2rQzVfBWum+/hxk0oLgq/EEp1AFwqJI3IzMO1W66DgarEh1KVUTceL7qSmp04uxamTlo76d5t+EzkrrtERowQ2XfflsQVFRUivXqJXHSRPDXi37LP9svEGOf+Ms4rqYUeqkd9ICX+9PUBf5baHs26Shoz3MhoGzwiIm+LSDcR6SkiPbF89fcVkc+B+4GBxpi2xpgdgV2Al71+VjnTo0d+5X5+3nvszuE8iUF4isP4Pu8B0L499O0LPXtCmzbWY329+3qdyp3OMSb9dfv2MG5c9s9T0jHGdDTG3G2MWWCMec8Yc7AxprMxZo4xZlHisVPY7YwDGzfmV57G+PFwySUwYIAlNJWVvrZNiR/GmP7GmHeNMc3GmF4Z740xxiw2xiw0xhwddFvq6mDKFKittfrd2lrrdV1dyznjxll9cCp2fXJ9fX46AuCjj+zLGxryqydM3PyGQePltwerjUuWQHOz1W472tZuA6ecAtddB6++Cl9+CY88AqNHw+abw4038rNJJ/Pa0m407/YDFhx6Dr+q/hc78j9AANXhSoE8/jjstx8sWcKTox7ixw+OoeEjg4jVVwwdGv1+QnFJLivW7UGKu2/i9VisrL4LgWPd1KGzS60p9qys3eftVfVf+dxsLZ/TTXp3f9dTMiM3blxO33XYsPKKJSCY1PnTgF8lnlcDHdFNyD3heSU1uYLav7+uoMYcP2UU+AGwG/AU0CulPLJZ8nPFd3nVW25CPuKyKhkWoW8ls26dyPPPi1x1lchxx4l07LipgqVsKw+0HyAvn/U3kTfeENmwwfP3zEUQetSvQ/WoR5qbrWz4bdqI7LGHyKJFJeF9Ua64kdHQhTX1UMG1p9gB37af9957IttsI9KtmxzZ/Z28OwU/U4CXOn4rV2BL4EPAZJRrVkIPeIqbUwO1pAhoIinTSI1t2IzXgaPbeMY4DkCLpdv8HLT70uaNG0XeekvkhhtEBg4U2W67lkZttZVI374if/6zyLPPWgauT6iRWmJ8+611/4DIKadsioF2GxKnY8vooUaqkpO8BHfBApHu3eULusrutDZUs8XJxjXjYRgEYKTujeVufxvwOnAzsDm6v5snpk9vvX9wmzZZZEcN1JKjSEZqbLPkF9Lfu4mLjJveKKZHVOR1bXOzFU/4r3+JnHOOyPe/39LItm1FDj1UZMwYkdmzRVav9vwxaqSWEP/7n8hee1k38V/+smn/UxF3kzJxjhMvZdzIqOeYVCX+1NdbvvsNDZbY5vTl3203ePJJpKKSJzmcPXgn7e3MmNLUuJg2DndaULG1ShqVwL7AjSKyD7AGy73XFSIyRUR6iUivrl27BtXG2DB2rBWzlUpzs1WeSn09jOtkxaA+2L4/M46boTGoZYox5nFjzDs2R79sl9mUiUP9Q40x840x85cvX+5PowugkFwKbuIi46Y3xo6FtWvTy9aubd1n+EGx81jkjTHWwODMM61g3ffeg2XL4N574bzzYN06uOYaK/FF586wzz5w4YUwaxZ89lnYrVeKzZw50KuXFbA+e7YV+5ySqMRNjHwx5U/xFzVSyxhPgrvbbrx01VNsMFU8yeHsydtA604h0wC2SypTVaXJE4rEUmCpiLyUeH03ltH6hTGmO0DicVlI7YsVTsldUsvr6+HdweMZu/oSZtGfE9fO4JxhlZrMoUwRkSNFZE+b474sly0Fdkh5vT3wqUP9kZpIcptcqVj1hI2bPsMvYvmbde0KJ54IEybAyy/D6tVWcpzLL7cM1alT4dRTYdtt4Xvfg8GDrcGFUrqIWJMVxxxj/e+vvALHHNMqKRjkTlTmdKvoLRR91EgtY7wqzhNG7cr8a59iQ0VbnuAIju7+VqtOwc4AziQza68SDGJl3P7YGLNboqg38F+sLNyDEmWDgGwDZiWBm5WKztJ01AAAIABJREFUJeeP589NloF6OjPYSKXO3Cr5Etss+X5luI1Cplw/KGR1M99MvXV1MGgQVFRYrysqrNex+s06dIDeveGKK2DuXMtofeklKzv6nnvCgw+2tsSV0mHNGhg4EC67DH7xC5g3D773PUfvP2jxvliypPW9npSFTJzKlQiRyx+4mIf66ReXghMsLFoksv32IjU1Vqa+FNzueRrHBBhBQzDxbntjxZW+BfwH6IRuQu6J6dNFqqrS7+OqqpT4lkQM6p30lwqaohkXphSEnzIKnIS1aroe+AJ4NOU9zZJfAnhKtibeYun8jL/zmmzGzXUFJbJJiUl0Igg96tcRdxkNNAnRBx+I/PCHVqKHq6/OO/7UjmxjUCU83Mho6MKaesRdcOOGL8ps8WJbQ9XNVgI6aLdHlWu0mT5dpLo6/T6urk7ITcJAfaB9awNVJ2VKB5VRJR+8Dq69XOdXdl+v44Os/WOBdeeDymgwBPrfPfqoSKdO1vHoo63e9poUTLepiSZuZFTdfcsYX1ypdt4ZnnrKcr054gh44w3APi7Gjsgkc1AUl4wdC42N6WWNjZaLL5dcAv378/WNM2jbPj1JUuTjwhRFCQSvoTVervMr/tVrspkRI+z7xxEjCq9bCZ9A/jtJxJ8eeyxsv70Vf3rUUa1O69zZ/nKn8iSxjNNWAI1JLXtSMyna+fK7ImmoJuNIXn+9lQFcU2MlSkrFrpPIN/5GUYqN3WBvJFaSJAYMgBkzOP2sSg4+OP2cgw+OWVyYoii+4DUm1ct1fmX39WrsrlyZu7yYiaQUf/H9v0uNPz3lFCv+dOedPbfPjlKJbS9H1EhV/GGnnSxDdYstLEP1tdfSDOAVK+BXv8qezCHvLXEUJQQyB3sjGc94LuHB9gOsm7WykuHDrXwfqcydC8OHF6+diqJEA68rOV6uK2TVqFjbxkV+mxzFEV//uw8+sGZv774brr4aZs6EzTd3PH3VqvzKlRIglz9wMY84++krCT780HL079RJZP78TcVu4hg0bsACjaWJNKmJk0ZixaDOMgOkflrTpnMqKuzv5YqKEBuu+IbKqJIvQSYh8uuaTB2debiJPaypsb+2pib7Z2lMajzw7b975JGs8ad2eB0jFuN+U/LHjYzqSqriLz17WiuqW20FRx4J8+cD7uIY1AVIiQvGtKyg3skABlXWIxUtMah2+wJnK1cUpbTxGlrj5Tov1zhtG1dRkZ+L5KRJrUN7kh5U+extqUSTgl1nRaxV0759YYcdrDGiTfypHV69BDQGOr6okar4T9JQ7djRMlRfecWVAaouQEocGDsWLmhsMVDrqOe7pkoGDWoZhDntAaz7simKEkWcdHRzc/4G8q23puejaNPGikkVcb+3pRJdPOcy+fZbOPVUGD3ayt/wwgtWqFgen+vFQG5oyK9ciQ5qpCrBUFsLTz9tpV3r04fjt7bfgz7VAI1CBjZN3FTauPl/c53TvyHdQN2ItYK6cWPLIMzJSD3sMP++i6Ioil8ENUm8ejU0NaWX6SpWGbJ4sRV/es89cO21MGNG1vhTJ7wYyE6TwzppHH3USFWCo0cPa0W1poa7vurDT9u+lPZ2pgEadgY2TdxU2rj5f3OeM34819oYqJk0N9u3YfFif7+ToiiKH/g1SZzZhzqFOGgYTxnxyCOw//7w6afW81GjnGdyA0DDb+KLGqlKsCQM1eruXZjT5ihO3ObFrAaoL1vieETjFkobN/9v1nPGW/ugNhw4gHPaORuo2dCBmaL4j3rAFI5fk8ROsa2ZaBhPGSACf/mLFX9aW2vFn/bpU/Rm1NbmV65EBzVSleDZYQd4+mmqt+3KvWuOovmFFwM3QL0MWjRxU2njJi7F6b/u32AZqAwYQO1z9dx4U+WmwVw+LkM6MFMUf1EPmGjhRl8WO4xHsSfIyZ07p37LQx0GwP/9H/e1G8jMC1+AHXf07wPyIAqhZIo31EhVAiOtA/zJ9vz7wqdg662tTG4vvBDo53oZtGjiptLGTVxK586t3x+J5eLLgJZ9UFNX/KdNa60Aq6tbZ7hUpago/qMeMP7gl7HvpC/zzRKsBEuQkzv3TVjMD885iGPW/ptRXMuJa+s5+4L2oU0chR1KphRArj1qinnEee8oJR2nfanuuX6pyC67iHToIPL887bXedlLLpW476WF7u8WCNn2/0uSucdfch/Ue6sHiDQ1OdZtd9/6cS8r0aTcZTRK97Yx9jJtTHhtiiN+7VOeuo908qiqUj2aekRBjwa2L/3s2bK6TUdZQWfpzRzf97yPUt+jFIYbGQ1dWFOPKAiu4g9ZO8ClKYbqc89tusYvI7GQQUsUOkBVrsHgRimn3jtJA3UmA6QSZwNVKT/KWUajMpmXJLDBdpnhl7E/fbpIdXV6HdXVaqSmHlHQo76Pk5qbRcaNEzFGXmNv6cn/fJ84cup7hg0Lf9ym5I8aqUpo5OwAP/lEZNddLUP12WdFxL/BRtwHLapcg8HN4Dp576QaqBU0tbp3ojCZoYRHOcto1PrXqBnNcaXU9G85y6gb/PQ469bua2nY/2TrxWmnyW47rAnkHnBqc+Z4U+U/HriRUY1JVQIhZ3znttvCk0/CdtvBMcfAs8/6lrhIg+QVO9zEpYwbB5e2Sd8HtU1VZdq9o4lalHImagnmNN7MH/zSm24S1Cnh4/X/zowB/x6LeOK7g9julf9YGfDr67n8L+1b5WSoqip8DObUx4ikv9aY9NJBjVSlFX5kfHPVASYN1R12gGOP5ZRuz9jW1blzfu3RQYs9xpgKY8zrxpgHE687G2PmGGMWJR47hd1GJ4LKQvj88+n1trthPFc3p++DmrmdmyZqUcqZKCaYC3PrslLBL73pJkFdHDHG7GCMedIY854x5l1jzIhEeWz0aCpe/+9UQ7EvD/EK+7M1X3AUc2DkyE37n2bqTT+2Rc2nj9FdGUqEXEutxTyi4AJR7vjpOuXaJfKzz0S+/31pbLu5HNX2qbTPrqpqHd9S6q4cBOSmBFwMzAAeTLy+BhideD4auDpXHWHIqF/3pF09qUemi6+buFU/422U+BCUjPpxlFtMqtKaMMMRnPpXKF4brHb4K6NAd2DfxPMtgPeB3eOiR/2itlbEsFHG8ifZiBV/WsuHaToyKJdvu77HSR/HJbyrnHEjo6Er1NQjzoJbKoQWT/LZZyI/+IE0tW0vp2795CblmplttRw6oCAGwMD2wFzgiBQjdSHQXVoU8MJc9YQho0HHSuUyUDMNUKd7sqbGxy+tRJpyNlJFNCY7yoQ9iVAuManAfUCfuOhRv5h509fyn4qTREBup07asabV/RXkREVm3zNsmE6axRU3MqruvkoaocUbbbMNPPkklTv3ZOY3P6f5iadYsgRWrQqpPaXHdcClQHNK2dYi8hlA4rFbGA3LhV/3pNP5I0mPQd1IZatzdK9cRWlB3WujS9jhCOWQE8IY0xPYB3iJmOhRO/IOo3n/fU6deCDHy/38qdNEzuJ2utW2b+UmHKTLd2bfM3myhneVMmqkKmmEGm+09dbwxBNWb9m3LzzxRCTjn+KGMeY4YJmIvOrx+qHGmPnGmPnLly/3uXW58esesDvfjYGaOcBymjhxKlcUpbwIKobeDWEntnIb6xjmb1QIxpgOwD3Ab0Tk6zyuC1WPZpJ3AsAHH4T994fly2kz5zEuX3URzWJsJ6k2brSvwqm8UHTSrHRRI1VJI+xZ0PrHt2a/r5/k7e925rsjj+OiveaW/KxsEfgxcIIxZgkwEzjCGDMd+MIY0x0g8bjM7mIRmSIivUSkV9euXYvV5k34dU9m1uNkoHbokH2ApRMnip8YY641xiwwxrxljLnXGNMx5b0xxpjFxpiFxpijw2xnlImSwRN29u8o9E+5jIawfyOvGGOqsAzUehH5d6I4UD0a1L3tesW9uRmuvBJOOAF23hnmz4cjjshad21tfuWK4kguf+BiHnH20y8lwoo3So2l6cIyeYs9ZS2byXXHP15W8U8EGEsDHEZLTOq1pCd8uCbX9WHJqF/35LBhIhUVzjGolZW56w475ksJHz9lFDgKqEw8v5pE4hWspCxvAm2BHYEPgIpc9ZWbHo2aPIYdkxm138OOYvxGfutRwAD/Aq7LKA9Mjwb5X7pKAPjVVyInnmi9UVcnsmZN6O1WSgc3MhrIQNjrUW7KNc4EYchmKq4uLJM3+aF8ZzYTmTOn8A9wgZvvFbQRX0QjtQYrmdKixGPnXNfHWUaTijNbkqSqKnf/pyaOKW+CklHgJKxVGoAxwJiU9x4FDs5VR5xl1AthG4WZRCH7d5D9kx91F+M3CsBI/QkgwFvAG4mjb5B6NMh7O2fdCxaIfP/71qzuddeJNDfnVb/qSCUXaqQqgRDULJmd4qphubzBXiKbbSby6KP+fAEH3HyvYswQBmmkFnrEOXNobW3uLL5uBwCqgMubAI3UB4AzEs//nnyeeD0VOCVXHeWmR6NgFKYSNaPZT/zSf3FcSfXzcCujQd7bWf/LBx4Q2XJLkS5dRJ54ovAPUxQb3MioxqQqeRNU9kC7mJmVdOGX28+F3XazYiIefbSwD8mCm+8VdubEUqeQWKVcsTv9G6wY1Jmc6pgkCXInGKmvh8GD09s4eHD046mU8DDGPG6Mecfm6JdyzlhgA5C8k4xNVeJQf6SSshSTKMRgphJ2Xocg8Uv/lfJv5CeF3Nu59KFtgqt/NFP3wR/h+OPhe9+DV1+Fww8v9Gv4SpTiz5UikMuKLeZRbjPAcSWo2b2sM3srVojsvbdI27YiDz/suf5sq19uvlcc3ZT8PIKWUa8z7NOni1RXp19TXZ3yH19rraDewamOK6huP0v3SVX8llFgEDAPaJ9Spu6+Lohi/Fupelr4qf/iHDZT6BF0TKqn6776SqRfP+vks84SWbvWVRuLSRRlXfGOGxkNXVhTj3JTrnElSFedrIprxQqRffaxDNXZs/OuN1fn5uZ7qZtSsDLqdRCU1XBMGKj/rs5toLpReNmuV8oDP2UUOAb4L9A1o3wP0hMn/Y8ySJzkxXgpVaMwarjVf1H4P0pFj3r5LfMepyxYILLbblb86aRJecefFotSdqUvR9RIVQIh1NmslStF9t3XWiZ76CHXl7np3DQmNXwZ9aqEnIzGZAyqnHqqVGYxUPMZAKiRqvhspC4GPqYlGcs/Ut4bi5XVdyFwrJv64qxHdaUk2kRFR7qhnPVoXpO9990nssUWIl27ijz1VKDtKpSoxZ8rhVEUIxW4IKFA3yUl9XbCVWlx4r2j3dQVZ+VaboQ6U5pqqD7wgKtL3HZupZ7dt9AjaBn1OsDJZaBKU5Nvs7Dq7quUs4wGia6URJ9c+i8q/2E5y6ir/2DjRpErrrDe2G8/kYaGQNvkB1G5txR/CNxIBQ4HHgfaJl53Szzq/m6KJ1wbgKtWifTqZe0Z4sJQjZNhUc7KVcTbJEDm/5s0UP9dbRmoyXr9mOHPGf+qlDzlLqNBUSorJVFwdw2LqPyH5SyjOXXU6tUiJ5xgvRHR+FM7orJKr/iDGxktNLvvMOAqEVkPICLLEuX9gJkisl5EPkysqB5Q4GcpJU59PQwZkp41dcgQh+xtnTrBnDnwox/BySfDAw8Uvb1KMNTVwZIl0NxsPdbV5b5m0iSoqrKej8TK4jvLnMp3N02HyspN9bbKZjjFXf2Z7bvllvR6brkl/3oURUknapl6vVBIhvJSoBT+w1JAxOH1ggVw4IHw0ENw/fVw223Qrl2xm+cJv3S4Eh8KNVJ3BQ41xrxkjHnaGLN/onw7rBibJEsTZYriyIgR/9/evYdZVdd7HH9/nQuK11D0GDc5HuxEVmgjap7SEI0wsbwAOhaZhp7QBz0KiZOaJaVB3p6wxABJRhEUZbRRvGWXJ02l1FTyaOokaokg2nPsCRi+54+1tuyZ2Xtm39dae39ezzPP7PXbe6/57r3nt9f6rt8NNm3qWrZpU1Ce0W67BYnqqFFwwgnQ1pZ13xs25FcuydLcDIsWweW7BQnq3f0nseWmJZzy1foej8s3Ac7290qxHxHZphqWJqn1Zcqq4TNMupYW2Ly5a9nmzfDIeSth9OjgxOehh+Ccc4JsL0F07K0tfSapfazvVg98CDgEmAEsMzND67tJAdavz68cCBLV+++HAw6AE0+ElSszPkxXd5Mtl7XRmt+cS8vGGTBpEse+2zNBFZF4q4aWkmzrLPe1/nK1qIbPMOm6/68ZW/kOl3Ljui8Fa86vXg2HH17w/rVWqVRKn0mqu4919/0z/KwkaCFdEXYvfhzYCuwRlg9J281g4I0s+5/v7k3u3jRw4MDiX5HUnlSieuCBQaJ61109HqKru8mVU/e5uXNhRtCC2nDbEvb5j3odOEUSKOktJbogmvzPMOnS/9d24V1WchyX8l2W7/g1+M1vYMiQrM/tS9Td2ZUg15Ziu/veBYwBMLP9gEbgbaANmGxm/cxsODACeLzIvyVVbvfd8yvvYtddYdUqaGqCk06CO+/screu7iZXn93nwgR1ed0kvvz+ErZQn/XAqQOciJSTLohK1FL/g//JGh5nNOO4j/Mafsymny6E7bcvat9RdmePOkGWyis2SV0I/LuZPQssBaaErarPAcsIFii/D5jm7p1F/i2pcumT36Q0NATlOUklqgcdBBMnwh13dLlbV3eTqaOjl/K0FtSTO5fQybYuvt0PnDrAiUi56YKoRK25Gdqn3sUTNprd2EjzXg/RtGgazacWP/40yu7stT7euxYVlaS6+yZ3PzXs/nuguz+cdt9sd9/X3T/i7vcWH6pUu+ZmOOMMqKsLtuvqgu28Du677AL33RdMDjBpUpdEVa1oyZT6f+huhgUJKpMmcfz7XRPUlPQDpw5wIlIJmS6I6vgjFbF1K1xyCYdf82V2avooe722mmV/+2zJLpJE2Z291sd716JiW1JFSqa1FRYvhs6wzb2zM9jO+2CeSlQPPjhIVJcvVytagnVm6INxAXP4oc8IWsyXLGHQsMyTJKUfOMt5gNMJqIhko+OPVMTGjTBhAnzve3DaafDrX8PgwVkfXshxK8ru7BrvXXuUpEpslLSla+edg0T10EPh5JP53bnLCtq3ko/oDRvWdfsC5jCHmdzTf2LwgdTX53TgLNcBTiegItIb9eKQsluzJuhBtmoVzJsHCxb0Ov600ONWlN3ZNd679ihJldgoeUvXzjtDezsceijXvn0KE7ktr31XS/JhZkPM7JdmtsbMnjOz6WH5ADN7wMxeDH9/KOpYoeeFgfHjtx2YUgnq7XUTee8nQYIKuR04y3WA0wmoiPRG3RSlrO68M0hQ330XHn4YvvnNPtc/Lea4FdX8HhrvXXuUpEpsZGvRGjCgiNbMnXeGe+9ldb9P00ozk1ia09+Eqko+tgDnu/tHCdY0nmZmI4ELgYfcfQTwULgdqUwXBhYvhilTYPZu21pQNy1s7bEOal8HznId4HQCKiK9UTdFKYutW+Hii+H442HkyGD90898JqenJvW4pQkwa4uSVImN8eMzl7/zTpGtmTvtxKvz2nl0u8NopZnJ3Ar03YqW1C/x7tz9TXf/Q3j7H8AaYBBwHLA4fNhi4EvRRLhNtgsDg2+dw0UbZ8LEiXzx3Z4Jaq7KcYDTCaiI9EbdFKXkNm6EY4+Fyy+Hr38dfvWrXsefdqfjliSBklSJjfb2zOVbt3bdLqQ1c9LpO/H6/Hae6PcZlnAq5+xxS5+taNX4JW5m+wAHAL8H9nL3NyFIZIE9o4sskOkCwAVsS1BTY1DjRCegItIbdVOUUrr7h8/z8sDRbGp/gJYBP6H1cz/Le/1THbckCZSkSmzk00JZSGvm5NN35JD1v6DuiM9y3Yav0EzvzbHV9iVuZjsBdwDnuvt7eTxvqpk9aWZPrlu3rnwBEnTtTpcag3pXYzwTVNAJqIj0Td0UpRR+fe4KjvjWwfTf8h6f45d8f8NZTD3T8p4rQ8ctSQIlqRIb+bRQFtyaueOOcM89cPjh8NWvwpIlWR9aTV/iZtZAkKC2uvuKsPjvZrZ3eP/ewFuZnuvu8929yd2bBg4cWJmA2Zag3sZEztwpnglqik5ARUSk1FITCdZbJz/etYXPXnsCz7I/n2I1v+MwoPC5MnTckrhTkiqxkanlsrERGhq6lhXdmplKVI84IkhUb74560Or4UvczAxYAKxx96vS7moDpoS3pwArKx1bdxs2BL/TE9RmWln3TnwTVBERkVJLTST4bsc7tHEsZ7/3fW7kDI7gEd5gUJfHdnREFKRIGSlJlYrpa83RTC2XCxfCokVlaM3s3x/uvhvGjAmmjl28uO/n5Pg6Yugw4CvAGDN7KvwZD1wBHGVmLwJHhduRGjq0Z4LaSX2ixwGLiIjkq6UFhr//LE9wEGN5kDP5KVO5kU306/HYuroIAhQpMyWpUhHFrDlaSGtmTolk//7Q1gZHHgmnnQY33VTW1xEVd/+tu5u7f8LdR4U/7e6+3t2PdPcR4e8NUce64tCeCWqSxwGLiIgU4qCO23mMQ9iR/+MIHmE+Z2Z9bGdn1+0EXkwX6UFJqlRELmuOlioBzGs/qUR17NhgGvdFi3rsK/2Lfvr0qlk7NX7mzOHApTPpOHgiFw1tZavVJ3ocsIiIVD8zG2dmL5jZS2ZW/HrjnZ0waxbLOYk/8XE+xWoe5dO9PmXYsG23k3gxXSQTJalSEbmsOZpLIpuLvPezww6wciUcdRScfnrQx5jMX/Tr1/f9OqQAc+bAzGCZmWG/beUvHfWJHgcskisz+56ZPRN2w7/fzD6cdt+s8MT3BTP7fJRxikhPZlYHzAO+AIwETjazkQXv8P334Zhj4IorePFzUxm/wyO8yQdfCTQ0BHN1pOve26hU51IiUVOSKhWRy5qjuSSyucj2+I6OXrq/pBLVo48OEtUFCzJ+0WejMZNFSEtQ47rMjEgZzUl1xQfuAS4BCE90JwMfA8YB14cnxCISH6OBl9z9ZXffBCwFjit4bzvsAHvuCTfcwIiHb+DHN/brMifHokXBdfTe5uko1bmUSNR0NigVMXt20CqZnvR1v/o3dGjmGeryTQCz7cdsW3mq+wukfblvvz3cdRccfzyccQZHs5Ub+Uaff09jJouQSlAnTQqWA1KCKjWm25rFOwIe3j4OWOru/wJeMbOXCE6IH61wiCKS3SDgtbTttcDB3R9kZlOBqQBDezupMQsmcjQDgvOTTL2JeuthVKpzKZGoqSVVKiKXNUczLUFTSAI4fnzmcveu2xm7v2y/PaxYAePHM5+pfIP5Pfaz++7VsXZq5MIE9dVDJrHvo0vYrrFeEzxITTKz2Wb2GtBM2JJK5pPfQd2fKyKRsgxl3qMgn/XGLdMuc1eqcymRqClJlYrpa5beXBLZXLS35/7YjN1fwkT19VHHMJ8zmcoNH9zVvz9ce23y106NXFqC+omnl/DyX+s1wYNULTN70MyezfBzHIC7t7j7EKAVODv1tAy76nHyG+5/qpk9aWZPrlu3rjwvQkQyWQsMSdseDLwRUSxA6c6lRKKmvnUSK9m6tuQjn3EXWbu/9OvHoMfuYO2hJ3LDH8+ijq20D/tvZs/WF33R0rr4HvnoEv7xz65fQ6kWbr3PUi3cfWyOD70F+AVwKXmc/Lr7fAi6fTQ1NWVMZEWkLJ4ARpjZcOB1gnHkp0QbUmnOpUSippZUqTrZEs/uPWj67P7Srx+DH70djj2W6/kmr868Xl/6xeo2BvWV1zJfJ9MED1IrzGxE2uYE4M/h7TZgspn1C0+ARwCPVzo+EcnO3bcQ9H5YBawBlrn7c9FGJVIdlKRK1ck2HuOsswro/tKvH9x+O0yYANOmwbx5WiS7UBkmScpl1meRKndF2PX3GeBoYDpAeKK7DHgeuA+Y5u6d0YUpIpm4e7u77+fu+7q7Rn6KlIi6+0rVSSWeLS1Bi9zQoRTXTbexEZYvD5ZIOftsVjc4HZuDYWMZZwmWnrLM4pvLrM8i1czdT+jlvtmAaoOIiNQctaRKVco0SVNRLaCNjbBsGat2+BJXbT6Hc7jug7u0SHYfellmRhM8iIiIiEh3akmVmtDa2rXFrqAW0MZGJvzzNm5lMtcxHcO5LuiZpzGU2eSwDqomeBARERGRdGpJlZrQ0tK1SykU1gK697BGJnEbd3A813Iu07kG0BjKjHJIUEVERCR3mhdDaoWSVKkJ2Vo6820BnT0bGvs3MJml3M4JXMN5zGy4WmMou1OCKiIiWSjRKkyqV1hHB1pbXKqeklSpCaWaRTY1hnLQsAZO4VZ+0f9Ertz8PzT/7UfFB1ktlKCKiEgWSrQKV6peYSJJoCRVasL48fmVp2S62pualGmTN3DMxlvgpJPgggtg7twSR11+ZjbOzF4ws5fM7MKid6gEVUREeqFEq3Cl6hUmkgQ6g5Sa0N6eXznkONlSQwPcckswNe2MGcFl4RkzShZ3OZlZHTAPOApYCzxhZm3u/nxBO3SH119XgioiIlkp0Src0KHBuUimcpFqo5ZUqQmFHBRzvtpbXx9ktJMnB62IV15ZVKwVNBp4yd1fdvdNwFLguIL3ZgZXXx28F0pQRUQkg1INv6lFs2cHa4mn09riUq2UpEpNKOSgmFdiW18PN98MJ58MF14IV1yRd4wRGAS8lra9NiwrnBnU1RW1CxERqV5KtAqntcWllihJlZpQyJjUvBPb+nr4+c/hlFNg1iz4wQ/yijEClqHMezzIbKqZPWlmT65bt64CYYmICFTnLLhKtIqTmhdj69bgt943qVbqkyc1oZAxqbNndx2TCjlc7a2vh8WLgyPvRRcFR5H4zgaxFhiStj0YeKP7g9x9PjAfoKmpqUcSKyIipZfTvAgJ1dyc/NcgIuWlllSpCYWMSS34am8qUT31VPj2t+HyywuOu8yeAEaY2XAzawQmA20RxyQiImgWXBGpbWpJlZpQ6Ix4BV/trauDm24KstuLLw5aVC+5pID3F7ZMAAAIcUlEQVQdlY+7bzGzs4FVQB2w0N2fizgsERFBs+CKSG0rqiXVzEaZ2WNm9lQ4Zm102n2zwrUXXzCzzxcfqkjhIpmooa4OFi2CKVPg0kvhssvK+McK4+7t7r6fu+/r7pFMW1GNY65ERIqlWXBFpJYV2933h8Bl7j4KuCTcxsxGEnQd/BgwDrg+XJNRJBLNzUGumJp4tq4u2C77mJi6OliwAL72NfjOd4If+UBqzFVHR7DMamrMlRJVEal1mgVXRGpZsUmqA7uEt3dl26QrxwFL3f1f7v4K8BLBmowikWhtDYaJdnYG252dwXZFkqFUonraaUFr6sKFFfijyaAxVyIimWkWXBGpZcUmqecCc8zsNWAuMCssz3n9RS1vIZUQeTK03Xbws5/B3LkwcWKF/mj8acyViEh21bDciIZ0FE7vndSyPidOMrMHgX/LcFcLcCRwnrvfYWYTgQXAWHJcfxG0vIVURqZJk3orL4vttoPzz6/gH4y/AQNg/frM5SIikmzVvIxOuem9k1rXZ0uqu4919/0z/KwEpgArwocuZ1uX3pzWXxSplLosI6KzlYuIiEhxIu/FlGB676TWFdvd9w3g8PD2GODF8HYbMNnM+pnZcGAE8HiRf0ukYKmxqLmWS2Vs2JBfuYiIJIeGdBRO753UumKT1G8APzKzp4HvA1MBwrUWlwHPA/cB09xd6YBEZtiw/MqlMrTEgkjAzC4wMzezPdLKtJSbJJq+4wun905qXVFJqrv/1t0/5e6fdPeD3X112n2zw7UXP+Lu9xYfqkjhNJV/POlzEQEzGwIcBfw1rUxLuUni6Tu+cHrvpNYV25Iqkgiayj+e9LmIAHA1MJOuEwxqKTdJvGr4jjezOWb2ZzN7xszuNLPd0u4rW2+HanjvRIrR5+y+ItWiuVlf7nGkz0VqmZlNAF5396fNukyMPwh4LG2716XcCIfbDFVfQImZKviOfwCY5e5bzOxKguUWv9Wtt8OHgQfNbL9SDm+rgvdOpGBKUkVERMqoj6XcLgKOzvS0DGVayk2kwtz9/rTNx4ATw9sf9HYAXjGzVG+HRyscokhVUpIqIiJSRu4+NlO5mX0cGA6kWlEHA38ws9FoKTeROPo6cFt4O+feDiKSPyWpIiIiEXD3PwF7prbN7FWgyd3fNrM24BYzu4qgK6GWchMpk956O7j7yvAxLcAWoDX1tAyPz9iTQV3yRfKnJFVERCRm3P05M0st5bYFLeUmUjbZejukmNkU4IvAke6eSkRz7u2gLvki+dPsviIiIjHg7vu4+9tp21rKTSRiZjYO+BYwwd3fT7urDZhsZv3MbDjq7SBSUrbtglD0zGwd0NHHw/YA3u7jMXGTxJghmXFXQ8zD3H1gVMH0RnU0dpIYdzXErDpaeYq5cpIYd9nqaDghUj9gfVj0mLufFd7XQjBOdQtwbi4Xk1RHYyWJMUMy4867jsYqSc2FmT3p7k1Rx5GPJMYMyYxbMUcvia8niTFDMuNWzNFL4utRzJWTxLiTGHNvkvh6FHPlJDHuQmJWd18RERERERGJDSWpIiIiIiIiEhtJTFLnRx1AAZIYMyQzbsUcvSS+niTGDMmMWzFHL4mvRzFXThLjTmLMvUni61HMlZPEuPOOOXFjUkVERERERKR6JbElVURERERERKpUopJUMxtnZi+Y2UtmdmHU8WRiZgvN7C0zezatbICZPWBmL4a/PxRljN2Z2RAz+6WZrTGz58xselge27jNbHsze9zMng5jviwsj23MKWZWZ2Z/NLN7wu3Yx5yLJNRPUB2tFNXR+FEdLR/V0cpSHY2W6mhl1HodTUySamZ1wDzgC8BI4GQzGxltVBndBIzrVnYh8JC7jwAeCrfjZAtwvrt/FDgEmBa+t3GO+1/AGHf/JDAKGGdmhxDvmFOmA2vStpMQc68SVD9BdbRSVEdjRHW07FRHK0t1NFo3oTpaCbVdR909ET/AocCqtO1ZwKyo48oS6z7As2nbLwB7h7f3Bl6IOsY+4l8JHJWUuIH+wB+Ag+MeMzA4rJxjgHuS+P+R5XUlpn6G8amOVjZe1dHoX5fqaGXjVx0tX6yqozH4UR2teLw1V0cT05IKDAJeS9teG5YlwV7u/iZA+HvPiOPJysz2AQ4Afk/M4w67EjwFvAU84O6xjxm4BpgJbE0ri3vMuUhy/YQEfQaqo2WnOhpPifkMVEfLTnU0nhLzGaiOll1J6miSklTLUKapiUvIzHYC7gDOdff3oo6nL+7e6e6jCK7YjDaz/aOOqTdm9kXgLXdfHXUsZaD6WQGqo+WlOirFUh0tL9VRKZbqaHmVso4mKUldCwxJ2x4MvBFRLPn6u5ntDRD+fivieHowswaCStvq7ivC4tjHDeDuG4FHCMZHxDnmw4AJZvYqsBQYY2ZLiHfMuUpy/YQEfAaqoxWhOhpfsf8MVEcrQnU0vmL/GaiOVkTJ6miSktQngBFmNtzMGoHJQFvEMeWqDZgS3p5C0A8+NszMgAXAGne/Ku2u2MZtZgPNbLfw9g7AWODPxDhmd5/l7oPdfR+C/9+H3f1UYhxzHpJcPyHmn4HqaGWojsZarD8D1dHKUB2NtVh/BqqjlVHSOhr14Np8foDxwP8CfwFaoo4nS4y3Am8Cmwmuip0O7E4wgPjF8PeAqOPsFvN/EXQpeQZ4KvwZH+e4gU8Afwxjfha4JCyPbczd4j+CbYPJExFzDq8p9vUzjFN1tDIxq47G7Ed1tKwxq45WPn7V0ejiVB2tTMw1XUctfKKIiIiIiIhI5JLU3VdERERERESqnJJUERERERERiQ0lqSIiIiIiIhIbSlJFREREREQkNpSkioiIiIiISGwoSRUREREREZHYUJIqIiIiIiIisaEkVURERERERGLj/wE2k3KURnNUMwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1152x288 with 4 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "#-- Plot some example tiles\n",
    "fig,ax = plt.subplots(1, 4,figsize=(16,4))\n",
    "#-- loop over to plot the first 4 tiles\n",
    "for i in range(4):\n",
    "    ax[i].plot(coords[i,:,0],coords[i,:,1],'bo')\n",
    "    ax[i].plot(np.arange(w),params[i,0] + np.arange(w)*params[i,1],'r-')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Breakout (8 minutes)\n",
    "Try to create a neural network that learn from the examples above and get the slope and coefficient for a given linear feature.\n",
    "\n",
    "**HINT** You can use a series of Dense layers as before to find an approximate mapping between input and outputs. However, note that since we want to output regression values here, instead of a `softmax` activation, simply use the `linear` activation to preserve the output of the last layer instead of mapping the output to probability space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "\n",
    "---\n",
    "### My Solution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_2\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_4 (Dense)              (None, 128)               23168     \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 256)               33024     \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 256)               65792     \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 2)                 514       \n",
      "=================================================================\n",
      "Total params: 122,498\n",
      "Trainable params: 122,498\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model2 = keras.Sequential()\n",
    "\n",
    "model2.add(Dense(128, activation='relu', input_dim=coords.shape[1]*coords.shape[2]))\n",
    "model2.add(Dense(256, activation='relu'))\n",
    "model2.add(Dense(256, activation='relu'))\n",
    "#-- for final model we use a simple linear activation, which simply returns the input unmodified\n",
    "model2.add(Dense(2, activation='linear'))\n",
    "\n",
    "#-- we use a mean absolute error as the loss function\n",
    "model2.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])\n",
    "model2.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a side note, in addition to summarizing the model as above, we can also plot it. This is particularly useful as models get larger and more complex with less linear architectures such as skip-connections, parallel layers, etc. that are outside of the scope of this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZwAAAIECAIAAACi/g/dAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeVwT1/ow8JMAsoMssikivRdUWottEdkEfAVRBLECRSOCqMENrkap0IpaBUWsFaSiuGCrRbAsLUrdEETLIopWwKp04+pViUooBkgwYZn3j3N/c9MAMUA20uf7h5/kzHDmmUnyOMuZZygEQSAAAFAWVHkHAAAAkgRJDQCgVCCpAQCUCiQ1AIBSURV8c+PGjf3798srFAAAGIKNGzc6OzuTb/+yp/bkyZP8/HyZhwRAP/Lz858+fSrvKKSiurq6urpa3lEoifz8/CdPngi2qPadKS8vT1bxADAgCoXCYDA++ugjeQciecHBwQh+aBJCoVCEWuCcGgBAqUBSAwAoFUhqAAClAkkNAKBUIKkBAJQKJDWgPAoLCy0tLR8+fCjvQCSsuLi4qKjozJkzb7/9NoVCcXNz6+7uJqe2trYmJibq6upqampu3769paVF9hG+evUqPj7+k08+EWrPz8+PjIz85JNPFi9evG3btq6uLnLS7du3AwMDY2JiIiMjT548iRvr6uoOHjw43CobhIBvv/1WqAUAeUEIffvtt4P6k+Li4vfff7+xsVFKIREE0dTUNPxOgoKCgoKCxJz50KFDhw4dwq+bm5tVVVURQgwGQ2i2DRs2rFq1avixDcG5c+fwyJuoqCjB9m+//faDDz7o7u4mCKK3t9fX13fz5s14Um1trba2dmVlJUEQnZ2dtra2hw8fxpNu3rxJziaOvt8T2FMDysPb2/vOnTvW1tZS6r+1tTU0NFRKnfertLT06tWra9aswW+NjY1xUktJSSkoKBCc08rK6p///KcsYyP5+/sfO3asb/vRo0ednZ1VVFQQQhQKZc6cOWfPnsWTNm3aNH36dBcXF4SQhobG+vXrP/744/b2doSQo6Ojjo5Oenr6kOOBpAaAWPh8Po1Ga2xslNkSe3p6GAzGjh07BBttbW0DAgIQQsuXL//tt9/Idk1NTU1NTZnFJkRdXb1vY3t7e0lJCXmkXF9fP3bsWIQQk8ksLS11d3cn53R3d+/o6MjKysJvN27cuHPnzj/++GNowUBSA0qitbU1MzPT29u7sLAQIVRbW/vxxx+/9dZbra2ty5YtMzY2dnR0xCnpwYMHW7ZssbOza2pqWrBggaGhoaOjI75vKScnR09Pz9LSEiHU1taWmpqqoaGB7yvMy8u7f/8+i8Wi0+n79u1DCFVWVlpaWl68eFFKa5SZmclms+3s7AQbqVRqVlbWO++809bWFhgY2NnZ2e/fFhQUREVFxcTEzJ07Nz4+nsfjid4mCCGCIDIyMtasWTN9+vTZs2cLZsyhiYiIaGhooNFor1+/rq6uLikpwdvtwYMHCCEbGxtyTvy6qqoKv9XW1nZwcNi9e/cQFyx0DIzgnBpQDGiQ59QePHjAYDAQQvn5+QRBMJlMLy8vhNCqVavu379/5coVPT29RYsWEQQRFxc3evRoFRUVBoNRVlZWUFBgbGyspaWFz5fNnj173LhxZLcODg5OTk74tZ+f34QJE8hJ58+f19TUPH369GBXTcxzaj4+PsHBwUKNU6dOJQiisbHRyMgIIRQeHo7bMzIy8Cl2giBSUlJcXFz4fD5BECwWy8bGxsPDo7e3V8Q2IQgiKSnp66+/Jgiiu7vbycnJzMyMw+GIuUavX79Gfc6pEQSxbt06hNDkyZO9vb0fP36MGw8ePIgQ+uGHHwTnVFdX9/DwIN8mJCTo6+vj83Gi9f2ewJ4aUBKTJ0/Gx2WYmZnZtGnTEEK7du2ys7Pz8vKaMWPGnTt3EEJJSUm+vr5UKjU5OdnT03PhwoUZGRlcLjcjIwMhpKWlJdgtPofVL19f3/b2dhqNJqU1evjwIc5cfVlbW+fn56upqZ08eVLofNbLly/j4+NXr16tpqaGEDIyMvr000+vX79++vRpEdukqakpNTV16dKlCCEVFZWgoKDnz58XFRUNcxUOHDjg4ODQ0NBQXl5eUVGBG589e4YQ0tHREZxTR0fnxYsX5FtTU1M2m4336QYLkhpQHkIJCJ+iJht1dXXxqWiEkJaWloqKCv7ZI4QCAgLU1dXv3bs32CXiRUgDh8N58uSJgYHBQDN4enqmpaUhhKKjo3/66Seyvbq6msPhjB8/nmzx8/NDCJWVlaGBt0lVVVVXV9eqVavodDqdTm9oaFi5cuUwT9LxeLyAgIDIyMjLly/r6emFhoZmZ2cjhPDRPZfLFZyZy+UKxjx69GiEkGCaE9+A/wsB8PehqqpqYWEhOPhL7rq6ugiC6OnpETHP6tWr6+vrDx8+HBwcTKfTdXV1EUKPHz9GCP3555/kbOTBtYiuHj58qK2t3e9FzCFjMBidnZ10Oh0hVFNT4+HhsXbtWj8/P3yVls1mk3Py+fzOzs6JEyeSLVQqFSE0tKwKe2oAIIQQl8udNGmSvKP4H319fQ0NjVevXgm1E38dmJqWlubp6dnY2Lhr1y7cgke09L1KK3rttLS0nj59KlTAjsViDS14LDc3l6zdOH78+B07drDZ7Lt37+IhxI8ePSLnxK8FI8RJefLkyUNYLiQ1ABCTyWxubg4KCkIIqaqqdnR0kLtIHR0dvb29+DWVShUcE48QIidJHIVCcXFxEdq9IghC6HKnqqpqfn6+tbV1R0cHbnF2dtbT08OXgLGnT59yudz58+eLWNyUKVMIgoiNjSVbXr58+dVXXw1nFYyNjcnjfYSQg4MDQsjExMTCwsLd3f369evkpOvXr48aNSowMJBsYbFYZmZmhoaGQ1guJDWgPPAPHg9fQAjhBEQeVHZ2dgqex+HxeHV1dfh1YmJieHi4o6MjQmjKlCmvXr1KSkr69ddfExMTeTzeL7/8cvfuXYSQhYXF8+fPa2trr127xuVyS0pKDAwMpFcsmkajVVVVCe6aPXv2jMlkCiVWIyOjc+fOkefdjYyMkpOTKysrS0tLcUtaWlp4ePjMmTPRwNvE29t72rRp2dnZgYGB33zzzfbt25csWRIREYEQWrt2rZub2++//y4iVA6HgxDC10BJkZGROTk5zc3N+G1xcfGMGTPwMebevXsrKirwVuXz+V9++WV8fLypqSn5t1VVVXPnzh3sFvsvwUuhMKQDKA40yCEdN27c8PX1RQi5u7tXVlaWlJTgczdr1659+fLlqVOn9PX1EUKfffZZd3f3ypUrR40axWAwgoODV6xYkZCQ0Nvbi/ths9n+/v46OjpOTk41NTXLli0LDQ09d+4cQRB1dXWWlpa2trZ5eXkEQVy9etXc3LywsHCwqybmkA4+n29jY4PzGkEQ3333nYeHB0IoKCiovLxcaObCwsL09HTy7dmzZ318fKKjo7du3bpv3z68dqK3SUtLy5IlS0xMTMaMGRMWFvbs2TPc1bx586hUamxs7EBxlpeXr1ixAiFkamqak5PDZDLJSUeOHPHx8YmJiYmNjV2/fn1LSws5qaamJiQkBN8WevDgQXL7EwTB5XINDQ0bGhreuImI/r4nkNSAghpsUhuUlStXamhoSKnzNxL/3s+ampr58+dLO543+vHHH/fs2SOzxcXHx3/++ediztz3ewKHnwAoLgcHBxqNdvz4cTnG0N7eXlRURN5/Km0XL17s6uqKiYkZcg+SSWrkSUoARoSOjg48ZkLegbxZSEiIlZXVpUuX5BVAfX39zp079fT0ZLCsuro6Npu9Z8+e4XQy3KR27Ngxb2/voV15lZ6rV69aWFiIM6fiVOAqKSlZuXIlhUKhUCg+Pj6nT5+W9hLz8/OdnJzwEtevX19bWyvtJSqIU6dOXblypaenZ9OmTbdu3ZJ3OG/m7e09Z84ceS3d1dVVQ0NDNsuyt7dftGjRMDsZ7uDb5cuXnzp1SqFGLXZ0dKxYsULM/4S1tbVNTEyk+pkxmUxzc/M3zubl5eXl5XXu3Lnm5uYTJ07gegZSjScoKGjcuHHOzs5Tp049cOCAlBangMLCwsLCwuQdBZCW4e6pqaiojBs3TiKhSMq2bduEChuIoGgVuPBOPr4mJYN48M0o0lscALKnbBcKrl27ZmpqKn5Sk6ohVODCT2bt+3xWKcUj1cUBIBdDTGpnz56NjIyMjY2Njo5mMplkO9FfSSbRVZxqa2sjIiKSk5MDAgK8vb1F9PNGHA7n0KFD4l83UfAKXDKI541evHgRGRmZkJBAp9M//PBDXP/+7Nmzurq6FAolNTWVz+cjhG7cuGFubo4LYPX72T1//nz//v3vvvsuk8mcPXu2lZWVXErpg78FwfEdYo5TO3369PTp0zs7OwmCaG5uHjNmjJmZGZ7Ub0km0VWcJk6cWFFRQRAEj8fz8/MT0c8bA9uwYUN9fT1BEDExMWRIIihgBS48NrKjo0M28TQ0NCCEPD09B4rH09MzJCQEv7a3tw8NDcWv4+LiEEI1NTX4LY/Hmz59On7d72d38eLFSZMmqaiofPbZZ5mZmY6OjuTYzoEgaY5Tk69BPaMAiNb3ezLopMbhcMzNzbOzs8mWhQsX4gzy7NkzU1PTnp4e3I73Bc6cOUMQBH7MDIvFwpPmzZtnY2NDEASfz6dQKAcOHMDtly5dEt2PCNeuXUtISMCvxUxq+K/IpCYiToIgaDSampoaLrxHEAS+OWbbtm0EQSxYsEAwiTg5OQ2URAiCEF33TjCpySCeNya1mTNn7t69G79esmTJu+++i18/efJEVVV15cqV+O0PP/yAN76Izw4POv/tt99ErL4gSGpAHH2/J4O++lleXs5kMqdMmUK2jBo1Cr8gSzKRk8iSTANVcVJTU5s9e/aGDRt+/vnnPXv2+Pj4iO5nIBwOJy0t7cyZM4NdHQWvwCWDeES7evUqQojD4WRlZdXU1JD3b48bNy44ODgrKyspKcnY2Dg3N3f79u1I5Genpqamqqo6qIeDhISEhISESHJ9FAmcypSSQSc1/H87mcgEDa0k05kzZxYvXnzs2LHvv/8+Nzd35syZQ+gnPj7ez8+PrJP58uXLrq6uuro6TU1NW1vbQcUjJkWrwCWleHp6epKTk3/77beNGzdWVFTg03YYg8HIyck5evRoTEwMi8V66623kKTLcm3YsIGsXaNMUlJSEEL41AcYpr7/7Q06qeF09vjx477JgizJJDjIg8ViGRsbi+hQS0vr4sWLp0+fjomJmTNnTm1t7RD6qa6uTk1NFWqcOnXq1KlTcSUAaVC0ClySjef333+3sLD48MMPTUxMvvnmm74zTJs2zdXVNT09fdKkSf7+/rhxaN+BgTg7O+MHSiqZvLw8hJBSrprs9U1qg776+e677yKE8Nk3rLe3FxefGkJJJh6Pd/ToUYTQkiVLqqurCYIoKysbQj83btwQPKiOi4vD59Skl9GkVIELnyMgBn/7ztDiEbGgTZs23b17t7i42NPTE7f0va9o8+bNTU1NmzZtCg4Oxi3SKMsFwKAMOqm5urrOnDnz66+/Pnz4MJfLrampqaioaG5uzsnJcXV1Hagkk4jKVidOnMA/PwsLC319/ffff19EaSeJU7QKXG1tbUig0rG048ELEiqvymazw8PD8SkwhNDJkyfv3bt34sSJ+/fvv3jxor6+nqwc7+/v/84779jb25PPBxHx2XV3d/f09CjOATtQWoI7OGIO6WCz2cuXLzc1NR0/fvxnn30WGRkZERFRUlLS09PTb0kmEVWcOBzOtGnTfHx89uzZExkZefz4cbyIgUo7iYncUxNNoSpwlZWVrV27Fn8oc+bMOXPmjLTjKSwsdHNzw0t0cnLy8fHx9vaeNGkSPsNw5MgRgiBWr16tq6vr5ORUUlJy4cIFY2PjoKAg8uIsQRDr16/Hq0bq97PLysrC92atX7/+559/FudDRHD1E4ih7/cE6qmJS74VuPpSkHhmzZqFRyxKHCQ1II6+35OR9DSpMWPGDDTpxIkT5LlqifwVEEdZWdkHH3wgsxIOAIhjJCU1sti5DP6qL7ICl4IML5JjPBUVFatWrXr77bd//vnnH3/8UcZL/xsqLi7m8XgcDichIeHBgweurq7Xrl0jRy+2tramp6cnJyd3d3dv3rz5X//610CPQJaeV69e7du3r6enJykpSbA9Pz+/uLjYyMjo0aNHNjY2W7duJUdW3r59Oykpydrauq2tzdXVNTw8HCFUV1dXXl6+bt26YX2rBXfb4PBzICdPnsRfFAaDcfPmTXmHI+d4Hjx48NZbb/3jH//48ccfpbcUJM3DT3w/mbw6GdTh56FDhw4dOoRfNzc341zGYDCEZtuwYcOqVauGFs8wnTt3Dg9PiYqKEmz/9ttvP/jgA3wLTW9vr6+v7+bNm/Gk2tpabW3tyspKgiA6OzttbW0PHz6MJ928eZOcTRx9vyeQ1ICCkl5S+/PPP//f//t/cuxE/KRWUlIiNCd5sE/e24elpKSIX9df4vBldKGkNmvWLMGWtLS0iRMnkpMEt156erqOjk5bWxt+u3PnzoMHD4q56L7fE2UrPQSAaEOoByWlTt6op6eHwWDs2LFDsNHW1jYgIAAhtHz5csHqNZqamkN7nrlEqKur921sb28vKSkhB/HU19fj0qdMJrO0tNTd3Z2c093dvaOjIysrC7/duHHjzp07//jjj6EFA0kNjGwFBQVRUVExMTFz586Nj4/HQw7Fr78kl6JSYsrMzGSz2ULFAalUalZW1jvvvNPW1hYYGCj0bGPRm0V0ETBiSPW+RIiIiGhoaKDRaK9fv66uri4pKcHbCt/OaGNjQ86JX1dVVeG32traDg4OuJLVUAjutsHhJ1AcSIzDz5SUFBcXF1yqhMVi2djYeHh44PF6YtZfkllRKUFiHn76+PgEBwcLNU6dOpUgiMbGRnxSNTw8HLdnZGSQh2wDbRbRRcCGVu8Lw48xFjr8JAhi3bp1CKHJkyd7e3s/fvwYNx48eBAh9MMPPwjOqa6u7uHhQb5NSEjQ19cXXdIG6/s9gT01MFK9fPkyPj5+9erV+IKakZHRp59+ev36dfzMGi0tLcGZhcqxkJKSknx9falUanJysqen58KFCzMyMrhcbkZGhvidIIR8fX3b29tpNNrw14v08OHDga5jWltb5+fnq6mpnTx5Uqh8gIjNYmZmNm3aNITQrl277OzsvLy8ZsyYcefOHYRQU1NTamrq0qVLEUIqKipBQUHPnz8vKioa5iocOHDAwcGhoaGhvLy8oqICNz579gwhRD5SHtPR0SHvVEEImZqastlsskTFoEBSAyNVdXU1h8MZP3482eLn54cQKisrG1Q/cikq9UYcDufJkycGBgYDzeDp6ZmWloYQio6O/umnn8h20ZtloGJWZM0oOp1Op9MbGhreWO/rjXg8XkBAQGRk5OXLl/X09EJDQ7OzsxFC+Ihe8IY//FYwZvz0DME0J76RNE4NAEGPHz9GCP35559kC3nkOJxuFaSoFB6ESFYl6Nfq1avr6+sPHz4cHBxMp9N1dXXRUDeLZGtGYQwGo7Ozk06nI4Rqamo8PDzWrl3r5+eHb/4jb3BGCPH5/M7OzokTJ5ItVCoVITS0rAp7amCkws8A63sJcvj1lxShqJS+vr6GhoZQrQHUp7BKWlqap6dnY2Pjrl27cMvQNgtZM0qwkcViDS14LDc3lyyHN378+B07drDZ7Lt377799tsUCuXRo0fknPi1YIQ4KQ/tgcKQ1MBI5ezsrKenhx+agz19+pTL5c6fPx8Nsh6UICkVlRosCoXi4uIitHtFEITQ5U5VVdX8/Hxra+uOjg7cInqzDEQaNaOMjY3JQs0IIQcHB4SQiYmJhYWFu7v79evXyUnXr18fNWpUYGAg2cJisczMzAwNDYewXEhqYKQyMjJKTk6urKwsLS3FLWlpaeHh4TNnzkSDqb+EZFJUaghoNFpVVZXgrtmzZ8+YTKZQMjUyMjp37hx53l30ZhmomJWImlFr1651c3P7/fffRYTK4XAQQvgaKCkyMjInJ4e8T7G4uHjGjBn4GHPv3r0VFRV4S/L5/C+//DI+Pt7U1JT826qqqrlz5w52i/2X4KVQGNIBFAcS746Cs2fP+vj4REdHb926dd++fUOoByXtolJ9iTmkg8/n29jY4LxGEMR3333n4eGBEAoKCiovLxeaubCwMD09XfRmEV3MaqB6X/PmzaNSqbGxsQPFWV5ejp+qY2pqmpOTw2QyyUlHjhzx8fGJiYmJjY1dv359S0sLOammpiYkJOSTTz5ZvHjxwYMHyW1OEASXyzU0NGxoaHjjJiLgNikwgoiZ1IZP9kWcxL9NqqamZv78+dKO541+/PHHPXv2yGxx8fHx4t/y1fd7AoefACguBwcHGo12/PhxOcbQ3t5eVFS0Zs0a2Szu4sWLXV1d4j+SvC9IauDvjiziJO9A+hcSEmJlZXXp0iV5BVBfX79z5049PT0ZLKuuro7NZu/Zs2c4ncA4NfC3durUqStXrvT09GzatGnRokX4+oCi8fb2luPSXV1dZbYse3t7e3v7YXYCSQ38rYWFhYWFhck7CiBJcPgJAFAqkNQAAEoFkhoAQKlAUgMAKJV+LhTk5ubKPg4A+rpx44a8Q5AKfN84/NCkRXAkLr6jAAAARhChOwooCjvmECgT/Ag12DcBMgDn1AAASgWSGgBAqUBSAwAoFUhqAAClAkkNAKBUIKkBAJQKJDUAgFKBpAYAUCqQ1AAASgWSGgBAqUBSAwAoFUhqAAClAkkNAKBUIKkBAJQKJDUAgFKBpAYAUCqQ1AAASgWSGgBAqUBSAwAoFUhqAAClAkkNAKBUIKkBAJQKJDUAgFKBpAYAUCqQ1AAASgWSGgBAqUBSAwAoFUhqAAClAkkNAKBUIKkBAJQKJDUAgFKBpAYAUCqQ1AAASkVV3gEA5fTjjz/euHGDfNvQ0IAQSk5OJlucnZ3d3d3lEBlQdhSCIOQdA1BCpaWlXl5eampqVKrw0UBvb29XV1dJScmsWbPkEhtQbpDUgFT09vaamZk1Nzf3O9XY2Pj58+cqKioyjgr8HcA5NSAVVCp1yZIlo0aN6jtp1KhRoaGhkNGAlEBSA9KyePFiPp/ft53P5y9evFj28YC/CTj8BFI0YcKEx48fCzVaWlo+fvyYQqHIJSSg9GBPDUjR0qVL1dTUBFvU1NSWLVsGGQ1ID+ypASlqaGiYPHmyUOPPP//89ttvyyUe8HcAe2pAiiZNmvT2228L7pfZ2dlBRgNSBUkNSFdYWBh5oVNNTS08PFy+8QClB4efQLqePHliZWWFv2YUCqWxsXHChAnyDgooM9hTA9JlaWk5ffp0KpVKpVKnT58OGQ1IGyQ1IHVLly6lUChUKnXp0qXyjgUoPzj8BFLHYrHMzMwQQk1NTSYmJvIOByg7QjEEBQXJe0sAAIYuKChI3lnkvxSo9JCTkxODwZB3FOAvUlJSEELD/1x+/PFHCoUyY8YMSQQlGTdu3EhNTf3222/lHYgywN8TBaFASW3cuHEfffSRvKMAf5GXl4cQGv7nMnfuXISQrq6uBGKSnNTUVPjKSQT+nigIBUpqQIkpWjoDSgyufgIAlAokNQCAUoGkBgBQKpDUAABKBZIakLDCwkJLS8uHDx/KOxAJKy4uLioqOnPmDK474ubm1t3dTU5tbW1NTEzU1dXV1NTcvn17S0uL7CN89epVfHz8J598ItSen58fGRn5ySefLF68eNu2bV1dXeSk27dvBwYGxsTEREZGnjx5EjfW1dUdPHiQGLHD8uHqJ5AwbW1tExMTDQ0N6S2CyWSam5tLr/++Dh8+jBBas2YNQsjLy8vc3LyysnLz5s379+/HMxgYGMTHx7e0tHR2du7YsUOWsWFFRUVZWVm5ublRUVGC7bm5uXv37r1586aKigpBEH5+fvHx8fhZhXV1dZ6ensXFxS4uLq9fv7a3t+/s7Fy9erW9vT2Px4uLixN8pOEIAntqQMK8vb3v3LljbW0tpf5bW1tDQ0Ol1Hm/SktLr169ijMaQsjY2FhVVRUhlJKSUlBQIDinlZXVP//5T1nGRvL39z927Fjf9qNHjzo7O+PqTxQKZc6cOWfPnsWTNm3aNH36dBcXF4SQhobG+vXrP/744/b2doSQo6Ojjo5Oenq6DNdAYiCpgZGEz+fTaLTGxkaZLbGnp4fBYAjtfNna2gYEBCCEli9f/ttvv5HtmpqampqaMotNiLq6et/G9vb2kpIS8ki5vr5+7NixCCEmk1laWir4PGl3d/eOjo6srCz8duPGjTt37vzjjz+kH7iEQVIDktTa2pqZment7V1YWIgQqq2t/fjjj996663W1tZly5YZGxs7OjrilPTgwYMtW7bY2dk1NTUtWLDA0NDQ0dGxuroaIZSTk6Onp2dpaYkQamtrS01N1dDQcHZ2Rgjl5eXdv3+fxWLR6fR9+/YhhCorKy0tLS9evCilNcrMzGSz2XZ2doKNVCo1KyvrnXfeaWtrCwwM7Ozs7PdvCwoKoqKiYmJi5s6dGx8fz+PxRG8ThBBBEBkZGWvWrJk+ffrs2bMFM+bQRERENDQ00Gi0169fV1dXl5SU4O324MEDhJCNjQ05J35dVVWF32prazs4OOzevXuYAciBnO89/T9BQUGKc0MsIA32c3nw4AG+UTQ/P58gCCaT6eXlhRBatWrV/fv3r1y5oqent2jRIoIg4uLiRo8eraKiwmAwysrKCgoKjI2NtbS0mpqaCIKYPXv2uHHjyG4dHBycnJzwaz8/vwkTJpCTzp8/r6mpefr06cGuGr7r842z+fj4BAcHCzVOnTqVIIjGxkYjIyOEUHh4OG7PyMjAp9gJgkhJSXFxceHz+QRBsFgsGxsbDw+P3t5eEduEIIikpKSvv/6aIIju7m4nJyczMzMOhyPmGr1+/RohFBUVJdS+bt06hNDkyZO9vb0fP36MGw8ePIgQ+uGHHwTnVFdX9/DwIN8mJCTo6+t3d3e/cdEK9fuFPTUgSZMnT8bHZZiZmdm0adMQQrt27bKzs/Py8poxY8adO6HYrg4AACAASURBVHcQQklJSb6+vlQqNTk52dPTc+HChRkZGVwuNyMjAyGkpaUl2C0+h9UvX1/f9vZ2Go0mpTV6+PAhzlx9WVtb5+fnq6mpnTx5Uuh81suXL+Pj41evXo0fpmVkZPTpp59ev3799OnTIrZJU1NTamoqrjqnoqISFBT0/PnzoqKiYa7CgQMHHBwcGhoaysvLKyoqcOOzZ88QQjo6OoJz6ujovHjxgnxramrKZrPxPt0IAkkNSJhQAsKnqMlGXV1dfCoaIaSlpaWiokI+Qy8gIEBdXf3evXuDXaL0HvbO4XCePHliYGAw0Ayenp5paWkIoejo6J9++olsr66u5nA448ePJ1v8/PwQQmVlZWjgbVJVVdXV1bVq1So6nU6n0xsaGlauXDnMk3Q8Hi8gICAyMvLy5ct6enqhoaHZ2dkIIXx0z+VyBWfmcrmCMY8ePRohJJjmRgQY0gEUhaqqqoWFheDgL7nr6uoiCKKnp0fEPKtXr66vrz98+HBwcDCdTse37uNHOP/555/kbOTBtYiuHj58qK2t3e9FzCFjMBidnZ10Oh0hVFNT4+HhsXbtWj8/P3yVls1mk3Py+fzOzs6JEyeSLVQqFSEkx0sfQwN7akCBcLncSZMmyTuK/9HX19fQ0Hj16pVQO/HXgalpaWmenp6NjY27du3CLXhES9+rtKLXTktL6+nTp0+fPhVsZLFYQwsey83NxddYEELjx4/fsWMHm82+e/cuHkL86NEjck78WjBCnJT7PrlVwUFSA4qCyWQ2NzfjGsiqqqodHR3kLlJHR0dvby9+TaVSBcfEI4TISRJHoVBcXFyEdq8IghC63Kmqqpqfn29tbd3R0YFbnJ2d9fT08CVg7OnTp1wud/78+SIWN2XKFIIgYmNjyZaXL19+9dVXw1kFY2Nj8ngfIeTg4IAQMjExsbCwcHd3v379Ojnp+vXro0aNCgwMJFtwHXZDQ8PhBCB7kNSAhOEfPB6+gBDCCYg8qOzs7BQ8j8Pj8erq6vDrxMTE8PBwR0dHhNCUKVNevXqVlJT066+/JiYm8ni8X3755e7duwghCwuL58+f19bWXrt2jcvllpSUGBgY5OfnS2l1aDRaVVWV4K7Zs2fPmEymUGI1MjI6d+4ced7dyMgoOTm5srKytLQUt6SlpYWHh8+cORMNvE28vb2nTZuWnZ0dGBj4zTffbN++fcmSJREREQihtWvXurm5/f777yJC5XA4CCF8DZQUGRmZk5PT3NyM3xYXF8+YMQMfY+7du7eiogJvVT6f/+WXX8bHx5uampJ/W1VVhat7jjByvfb6Pwp1SRiQBvu53Lhxw9fXFyHk7u5eWVlZUlKCz92sXbv25cuXp06d0tfXRwh99tln3d3dK1euHDVqFIPBCA4OXrFiRUJCQm9vL+6HzWb7+/vr6Og4OTnV1NQsW7YsNDT03LlzBEHU1dVZWlra2trm5eURBHH16lVzc/PCwsLBrpqYQzr4fL6NjQ3OawRBfPfddx4eHgihoKCg8vJyoZkLCwvT09PJt2fPnvXx8YmOjt66deu+ffvw2oneJi0tLUuWLDExMRkzZkxYWNizZ89wV/PmzaNSqbGxsQPFWV5evmLFCoSQqalpTk4Ok8kkJx05csTHxycmJiY2Nnb9+vUtLS3kpJqampCQEHxb6MGDB8ntTxAEl8s1NDRsaGh44yYiFOz3qyhPkwoODkYKVhQYICl/LnQ6PSsra6CRq9KWm5sbEhIizvf/9u3bCQkJ5N1F8lJeXl5VVSV4cCpVW7du1dfXj4mJEWdmhfr9jvjDT/IsBgBS4uDgQKPRjh8/LscY2tvbi4qKyPtPpe3ixYtdXV1iZjRFM4KT2rFjx7y9vRXt0szVq1ctLCzEmfPChQv+/v4UCgWfjXZzc3vvvfecnJxiY2NH4g13Q9DR0YHHTMg7kDcLCQmxsrK6dOmSvAKor6/fuXOnnp6eDJZVV1fHZrP37Nkjg2VJwwhOasuXL3/9+rVCDWvq6OhYsWKFmL9SX19fPHreysqqqqoKn7L98ssv6+vrJ06cuGXLFuld1FMEp06dunLlSk9Pz6ZNm27duiXvcN7M29t7zpw58lq6q6urVKs5CbK3t1+0aJFsliUNIzipqaiojBs3Tt5R/MW2bduE7nwWTVtbG/11cOO0adPOnz+/aNGi3bt3j9BqVmIKCwtjsVgEQezfvx9f8QRAIkZwUlM0165dMzU1HVRSo1AofRupVGp6erqJiUliYuJ//vMfyQUIwN/CyEtqZ8+ejYyMjI2NjY6OZjKZZDvRX80W0WVeamtrIyIikpOTAwICvL29RfTzRhwO59ChQ31PrA6tMI6+vv5HH33E5XJzc3PlvmoAjDByG0zyV2KOczl9+vT06dM7OzsJgmhubh4zZoyZmRme1G/NFtFlXiZOnFhRUUEQBI/H8/PzE9HPGwPbsGFDfX09QRAxMTFkSMSbCuPg+28mTZrUdxKu1RcRESHfVVOo8UeSJeY4NSAOhfqeKMqHKs5G4XA45ubm2dnZZMvChQtxBnn27JmpqWlPTw9ux2Xwzpw5QxAEfg4FPn1DEMS8efNsbGwIguDz+RQK5cCBA7j90qVLovsR4dq1awkJCfi1UFIjCEJEOSoRSe3y5csIoVmzZsl31RTqyypZkNQkSKG+JyOpSkd5eTmTyZwyZQrZMmrUKPyCrNlCTiJrtgxU5kVNTW327NkbNmz4+eef9+zZ4+PjI7qfgXA4nLS0tDNnzgw0w9AK4+DyCba2tnJcNezp06f4KFjJ3LhxAyGklKsme0+fPlWcq3YjKak1NDQggUQmaGg1W86cObN48eJjx459//33ubm5M2fOHEI/8fHxfn5+ZCG9ly9fdnV11dXVaWpq2traDioeQXhl7e3t5bhqWHV1dUhIyGD/aqRQ4lWTMVyJQBGMpAsFOJ3hSlVChlazRUtL6+LFi1lZWaqqqnPmzHn48OEQ+qmurl6+fPnU/3Pq1KmWlpapU6cO59dCEEReXp6enp6fn58cVw1TnMMKyYLDTwlSnIyGRlZSe/fddxFC+LuI9fb24uo0Q6jZwuPxjh49ihBasmRJdXU1QRBlZWVD6OfGjRuCn25cXBw+p4aLHyCRhXGIAYbpfvHFF/fu3du3b9/YsWPluGoAjEQj6fDT1dV15syZX3/99QcffBAeHn7//v2Kiorm5uacnJz58+fjmi2vX79esGDB77//XlVVlZOTg0SWvjlx4sSaNWtUVFQsLCz09fXff//96dOnD9TP0JSUlAQGBmZmZvb7Xxm+cVWwFM/jx4+/+OKLgwcPrl+/HlcrJcvRKNqqAaCgZL2fOgAxr56w2ezly5ebmpqOHz/+s88+i4yMjIiIKCkp6enp6bdmi4gyLxwOZ9q0aT4+Pnv27ImMjDx+/DhexEC1X8RE7qlhIgrjXL582d/fH38Kbm5us2bN8vX1nTt3LoPBqK2tFZxTjqumUFe1JAsOPyVIob4nUHoIiKLEn4v4pYfAGynU92QkHX7K0ZgxYwaadOLECXKHCwAgd5DUxEJWQwYAKLiRdPUTADkqLi4uKio6c+YMfg6Tm5ubYNmr1tbWxMREXV1dTU3N7du3t7S0yD7CV69excfH4/tMBGVnZzs4OOjp6Tk6Op4/f15wUn5+fmRkJC7nvW3bNnzpqa6uDj9nXnahSxTsqQF5YjKZ5ubmitCJaIcPH0YI4cKzXl5e5ubmlZWVmzdv3r9/P57BwMAgPj6+paWls7Nzx44dUg2mX0VFRVlZWbm5uVFRUYLtKSkpV65cWbp06aNHj44ePerv719cXIzvGs7Nzd27d+/NmzdVVFQIgvDz84uPj09OTra3t+fxeHFxcSO0+BXsqQG5aW1tDQ0NVYRORCstLb169SpZStvY2BjfmpaSklJQUCA4p5WVFb4kLXv+/v59bxfp6Oi4devWhQsX1q9fn5KSUlpaSqFQPv/8czz16NGjzs7O+GY7CoUyZ84c8jkMjo6OOjo66enpslwFSYGkBuSDz+fTaLS+j/uVfSei9fT0MBgMoZ0vW1vbgIAAhNDy5csFKzhpamrK8Xnm6urqQi03b97ctm0b+dbJyem9994jn7PX3t5eUlJCHkTX19ePHTuWnHnjxo07d+4ciZXlIakBySgoKIiKioqJiZk7d258fDx+7mdOTo6enp6lpSVCqK2tLTU1VUNDAz8wPC8v7/79+ywWi06n79u378GDB1u2bLGzs2tqalqwYIGhoaGjo2N1dfWgOkFDLWAnQmZmJpvNFqr9SaVSs7Ky3nnnnba2tsDAwIEeiNXvNhFdCI+QaM27WbNmCT3EQ19ff8KECfh1REREQ0MDjUZ7/fp1dXV1SUkJ3oaYtra2g4PD7t27hxOAfMh1lNz/KNTgPUAS83NJSUlxcXHh8/kEQbBYLBsbGw8PD/wQydmzZ48bN46c08HBwcnJCb/28/ObMGECfh0XFzd69GgVFRUGg1FWVlZQUGBsbKylpdXU1CR+J8SbCtgJEnPwrY+PT3BwsFDj1KlTCYJobGw0MjJCCIWHh+P2jIwMfIpdxDYRXQhvaOX8MPwY46ioqIFm6O7uHjNmTGZmJtmybt06hNDkyZO9vb0fP34sNH9CQoK+vr6I2lkkhfr9wp4aGK6XL1/Gx8evXr1aTU0NIWRkZPTpp59ev3799OnTCCEtLS3BmclCSUKSkpJ8fX2pVGpycrKnp+fChQszMjK4XC5+No2YnSCEfH1929vbaTTa8NcLe/jwIc5cfVlbW+fn56upqZ08eVLofJaIbWJmZjZt2jSE0K5du+zs7Ly8vGbMmHHnzh2EUFNTU2pq6tKlSxFCKioqQUFBz58/LyoqktS6FBUVjR07Njw8nGw5cOCAg4NDQ0NDeXl5RUWF0PympqZsNpusQDNSQFIDw1VdXc3hcMaPH0+2+Pn5IYTKysoG1Y+WlpaKigrOAgihgIAAdXX1e/fuDTaeoRWw6xeHw3ny5ImBgcFAM3h6eqalpSGEoqOjf/rpJ7Jd9DYZqBAeWfOOTqfT6fSGhgYxa96Jg8/n7927Nzc3l9w+PB4vICAgMjLy8uXLenp6oaGh2dnZgn8yevRohNCLFy8kEoDMwJAOMFy4GNSff/5JtpBHjsPpVlVV1cLCQr6PQMSPJcWVYAayevXq+vr6w4cPBwcH0+l0XV1dNNRtMuSad+KIi4tLSkqysbEhWxgMRmdnJ66bUFNT4+HhsXbtWj8/P/LpolQqFf31aWcjAuypgeGytrZGCPW9BDlp0qRh9szlcoffyXDo6+traGjgquuCiL8OTE1LS/P09GxsbNy1axduGdo2GXLNuzc6dOiQu7u7h4eHYGNubi6+3oIQGj9+/I4dO9hsNlkyC/1fUla054W/ESQ1MFzOzs56enqFhYVky9OnT7lc7vz58xFCqqqqHR0d5M5OR0cHWWCOSqXiIez9YjKZzc3NuGTToDqR4EOgKRSKi4uL0O4VQRBClztVVVXz8/Otra1xLSn0pm0yECnVvMvOztbQ0FiwYAHZUl5ejhAyNjbGh72Yg4MDQsjExIRsYbFYZmZmhoaGwwxAxiCpgeEyMjJKTk6urKwsLS3FLWlpaeHh4TNnzkQITZky5dWrV0lJSb/++mtiYiKPx/vll1/w7oCFhcXz589ra2uvXbuGK8HxeLy6ujrcSWJiYnh4OH7OsfidlJSUGBgY5OfnS2rtaDRaVVWV4K7Zs2fPmEymUCY1MjI6d+6cjo6OONtkoEJ4ZO28wMDAb775Zvv27UuWLImIiEAIrV271s3NjRxi1i8Oh4MQwtdASRcuXPjyyy+7urqOHDly5MiRjIyMqKio+vp6hFBkZGROTg55X3NxcfGMGTMmTpxI/m1VVdXcuXMHvcnkTq7XXv9HoS4JA5L4n8vZs2d9fHyio6O3bt26b98+PJ6DIAg2m+3v76+jo+Pk5FRTU7Ns2bLQ0NBz584RBFFXV2dpaWlra5uXl0cQxMqVK0eNGsVgMIKDg1esWJGQkDCETkQUsBMi5pAOPp9vY2OD8xpBEN999x0+iAsKCiovLxeaubCwMD09XfQ2EVEIr7u7e6Cad/PmzaNSqbGxsQPFWV5evmLFCoSQqalpTk4Ok8kkCOLWrVt9z4ipq6u3tLTgvzpy5IiPj09MTExsbOz69evJdoIguFyuoaFhQ0PDGzcRoWC/X6inBkSR5edCp9OzsrIGGsgqceLXU7t9+3ZCQgJ5C5G8lJeXV1VVCR6cStXWrVv19fX7Pp+7Xwr1+4XDTwDewMHBgUajHT9+XI4xtLe3FxUVkfefStvFixe7urrEzGiKBpIaUBQdHR14CIW8A+lHSEiIlZXVpUuX5BVAfX39zp07ycEWUlVXV8dms/fs2SODZUkDjFMDCuHUqVNXrlzp6enZtGnTokWL8PUBheLt7S3Hpbu6uspsWfb29vb29jJbnMRBUgMKISwsLCwsTN5RAGUAh58AAKUCSQ0AoFQgqQEAlAokNQCAUlGgCwXV1dV4CB9QHLj2rFJ+Lvi+caVcNdmrrq52cnKSdxT/pShJjawWABSKpL6puCzalClTJNKbRIwbNw7fLQ+Gz8nJSXF+wopymxRQbh999BFCKDc3V96BAOUH59QAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNQAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNQAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNQAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNQAAEqFQhCEvGMASujUqVP79+/v6enBb1ksFkLI2NgYv1VRUdm4cWNYWJjc4gPKC5IakIpff/114sSJImb45ZdfbG1tZRYP+PuAw08gFba2tvb29hQKpe8kCoVib28PGQ1ICSQ1IC1hYWEqKip921VVVcPDw2UfD/ibgMNPIC1NTU2Wlpa9vb1C7RQK5cmTJ2PHjpVLVEDpwZ4akBYLCwsXFxcq9S/fMSqV6urqChkNSA8kNSBFS5cuFWqhUChw0RNIFRx+AilqbW01NTXt6uoiW1RVVZ8/f25kZCTHqIBygz01IEUGBgbe3t7k5QIVFRUfHx/IaECqIKkB6QoNDSWvFRAEERoaKt94gNKDw08gXVwu18jI6PXr1wghDQ0NFoulra0t76CAMoM9NSBdWlpaH374oZqampqa2ocffggZDUgbJDUgdTQaraurq6uri0ajyTsWoPxU5R3Af924cePJkyfyjgJIRU9Pj5aWFkEQbW1tubm58g4HSIWlpaWzs7O8o0AIIUQohqCgIHlvCQDA0AUFBck7i/yXouypIYSCgoLy8vLkHQX4i+DgYITQ8D+X69evUygUd3d3SQQlGbm5uSEhIQRcKJME/D1REAqU1IASmzFjhrxDAH8XkNSALAjdAQqA9MBXDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNSAhBUWFlpaWj58+FDegUhYcXFxUVHRmTNn3n77bQqF4ubm1t3dTU5tbW1NTEzU1dXV1NTcvn17S0uL7CN89epVfHz8J598ItSenZ3t4OCgp6fn6Oh4/vx5wUn5+fmRkZGffPLJ4sWLt23bhotE1dXVHTx4cOQOdoGrn0DCtLW1TUxMNDQ0pLcIJpNpbm4uvf77Onz4MEJozZo1CCEvLy9zc/PKysrNmzfv378fz2BgYBAfH9/S0tLZ2bljxw5ZxoYVFRVlZWXl5uZGRUUJtqekpFy5cmXp0qWPHj06evSov79/cXGxl5cXQig3N3fv3r03b95UUVEhCMLPzy8+Pj45Odne3p7H48XFxSUnJ8t+RYYP9tSAhHl7e9+5c8fa2lpK/be2tsq4flFpaenVq1dxRkMIGRsbq6qqIoRSUlIKCgoE57SysvrnP/8py9hI/v7+x44dE2rs6Oi4devWhQsX1q9fn5KSUlpaSqFQPv/8czz16NGjzs7OuNodhUKZM2fO2bNn8SRHR0cdHZ309HRZroKkQFIDIwmfz6fRaI2NjTJbYk9PD4PBENr5srW1DQgIQAgtX778t99+I9s1NTU1NTVlFpsQdXV1oZabN29u27aNfOvk5PTee+/9/vvv+G17e3tJSQl5EF1fXy/47IiNGzfu3Lnzjz/+kHLUkgdJDUhSa2trZmamt7d3YWEhQqi2tvbjjz9+6623Wltbly1bZmxs7OjoiFPSgwcPtmzZYmdn19TUtGDBAkNDQ0dHx+rqaoRQTk6Onp6epaUlQqitrS01NVVDQwPfLJ2Xl3f//n0Wi0Wn0/ft24cQqqystLS0vHjxopTWKDMzk81m29nZCTZSqdSsrKx33nmnra0tMDCws7Oz378tKCiIioqKiYmZO3dufHw8j8cTvU0QQgRBZGRkrFmzZvr06bNnzxbMmEMwa9asyZMnC7bo6+tPmDABv46IiGhoaKDRaK9fv66uri4pKcGbFNPW1nZwcNi9e/dwApAP+d56SgoKClKcG2IBabCfy4MHDxgMBkIoPz+fIAgmk4lP36xater+/ftXrlzR09NbtGgRQRBxcXGjR49WUVFhMBhlZWUFBQXGxsZaWlpNTU0EQcyePXvcuHFktw4ODk5OTvi1n5/fhAkTyEnnz5/X1NQ8ffr0YFft22+/Fef77+PjExwcLNQ4depUgiAaGxtxafLw8HDcnpGRgU+xEwSRkpLi4uLC5/MJgmCxWDY2Nh4eHr29vSK2CUEQSUlJX3/9NUEQ3d3dTk5OZmZmHA5HzDXClTijoqIGmqG7u3vMmDGZmZlky7p16xBCkydP9vb2fvz4sdD8CQkJ+vr63d3db1y0Qv1+YU8NSNLkyZPxcRlmZmY2bdo0hNCuXbvs7Oy8vLxmzJhx584dhFBSUpKvry+VSk1OTvb09Fy4cGFGRgaXy83IyEAIaWlpCXaLz2H1y9fXt729XXqV2h4+fDjQQxWsra3z8/PV1NROnjwpdD7r5cuX8fHxq1evVlNTQwgZGRl9+umn169fP336tIht0tTUlJqaih/BpaKiEhQU9Pz586KiIkmtS1FR0dixYwWfJH3gwAEHB4eGhoby8vKKigqh+U1NTdls9oMHDyQVgGxAUgMSJpSA8HloslFXV7e9vR2/1tLSUlFRwT97hFBAQIC6uvq9e/cGu8R+nwMvERwO58mTJwYGBgPN4OnpmZaWhhCKjo7+6aefyPbq6moOhzN+/Hiyxc/PDyFUVlaGBt4mVVVVXV1dq1atotPpdDq9oaFh5cqVkjpJx+fz9+7dm5ubS24uHo8XEBAQGRl5+fJlPT290NDQ7OxswT8ZPXo0QujFixcSCUBmYEgHUBSqqqoWFhaCg7/krquriyCInp4eEfOsXr26vr7+8OHDwcHBdDpdV1cXIfT48WOE0J9//knORh5ci+jq4cOH2trafS9iSkRcXFxSUpKNjQ3ZwmAwOjs76XQ6QqimpsbDw2Pt2rV+fn56enp4BlyGQI6XPoYG9tSAAuFyuZMmTZJ3FP+jr6+voaHx6tUroXbirwNT09LSPD09Gxsbd+3ahVvwiJa+V2lFr52WltbTp0+fPn0q2MhisYYWvKBDhw65u7t7eHgINubm5pK1asePH79jxw42m3337l1yBpyUhS41KD5IakBRMJnM5uZmXANZVVW1o6OD3EXq6Oggn7NHpVIFn46MECInSRyFQnFxcRHavSIIQuhyp6qqan5+vrW1dUdHB25xdnbW09PDl4Cxp0+fcrnc+fPni1jclClTCIKIjY0lW16+fPnVV18Ncy2ys7M1NDQWLFhAtpSXlyOEjI2NyVMBCCEHBweEkImJCdnCYrHMzMwMDQ2HGYCMQVIDEoZ/8Hj4AkIIJyDyoLKzs5PL5ZIz83i8uro6/DoxMTE8PNzR0REhNGXKlFevXiUlJf3666+JiYk8Hu+XX37BOxEWFhbPnz+vra29du0al8stKSkxMDDIz8+X0urQaLSqqirBXbNnz54xmUyhxGpkZHTu3DkdHR3ybXJycmVlZWlpKW5JS0sLDw+fOXMmGnibeHt7T5s2LTs7OzAw8Jtvvtm+ffuSJUsiIiIQQmvXrnVzcyOHmPWLw+EghPA1UNKFCxe+/PLLrq6uI0eOHDlyJCMjIyoqqr6+HiEUGRmZk5PT3NyM5ywuLp4xY8bEiRPJv62qqpo7d+6gN5ncyfXa6/8o1CVhQBrs53Ljxg1fX1+EkLu7e2VlZUlJCR5hv3bt2pcvX546dUpfXx8h9Nlnn3V3d69cuXLUqFEMBiM4OHjFihUJCQm9vb24Hzab7e/vr6Oj4+TkVFNTs2zZstDQ0HPnzhEEUVdXZ2lpaWtrm5eXRxDE1atXzc3NCwsLB7tqYg7p4PP5NjY2OK8RBPHdd9/hg7igoKDy8nKhmQsLC9PT08m3Z8+e9fHxiY6O3rp16759+/Daid4mLS0tS5YsMTExGTNmTFhY2LNnz3BX8+bNo1KpsbGxA8VZXl6+YsUKhJCpqWlOTg6TySQI4tatW33PiKmrq7e0tOC/OnLkiI+PT0xMTGxs7Pr168l2giC4XK6hoWFDQ4M4G1Ohfr+K8jBjSdXCB5Il1c+FTqdnZWUNNHJV2sR/RsHt27cTEhLIW4jkpby8vKqqSvDgVKq2bt2qr68fExMjzswK9fsd8Yef5FkMhfLvf/8bHwsAJeDg4ECj0Y4fPy7HGNrb24uKisj7T6Xt4sWLXV1dYmY0RTOCk9qxY8e8vb0V5NJMe3v76NGjKf9n4cKFb3wU+YULF/z9/fH8Li4ubm5u7733npOTU2xs7Ei84W4IOjo68JgJeQfyZiEhIVZWVpcuXZJXAPX19Tt37iQHW0hVXV0dm83es2ePDJYlDSN4nNrybcySNgAAIABJREFU5ctPnTqlIMOaMjMzAwMD33rrLfx29uzZb/wTX19fe3v7cePGWVlZVVVV4caamppt27ZNnDgxNjY2ISFBiZ9XcurUqStXrvT09GzatGnRokX4+oAi8/b2luPSXV1dZbYse3t7e3t7mS1O4kZwUlNRURk3bpzo60Gy0dPTc/bs2StXroi4m6dfeG9O8FTutGnTzp8/HxYWtnv3bh0dnb4F/5RGWFhYWFiYvKMASkhpdwRkqaCgoLa2dtGiRUePHm1raxP/DykUSt9GKpWanp5uYmKSmJj4n//8R3JhAvC3MPKS2tmzZyMjI2NjY6Ojo5lMJtlO9FezRXSZl9ra2oiIiOTk5ICAAPLgot9+RCsrK+NyuQUFBatWrbKzsysuLiYnDa0wjr6+/kcffcTlcnNzc+W7agCMPHIcTiJIzHEup0+fnj59emdnJ0EQzc3NY8aMMTMzw5P6rdkiuszLxIkTKyoqCILg8Xh+fn4i+nljYF1dXbdv3162bBmVStXQ0Hjw4AFuF10YB99/M2nSpL6TsrKyEEIRERHyXTWFGn8kWWKOUwPiUKjviaJ8qOJsFA6HY25unp2dTbYsXLgQJ7Vnz56Zmpr29PTgdlzr7syZMwRB4NNSLBYLT5o3b56NjQ1BEHw+n0KhHDhwALdfunRJdD9iKigooFAoH374IdkiohyViKR2+fJlhNCsWbPku2oK9WWVLEhqEqRQ35ORdKGgvLycyWROmTKFbBk1ahR+QdZsISeRNVsGKvOipqY2e/bsDRs2/Pzzz3v27PHx8RHdj5gWLlwYHBxcU1NDtgytMA6bzUYI2drayn3Vqqur8dBKJYPvG1fKVZO96upqJycneUfxXyMpqTU0NCCBRCZoaDVbzpw5s3jx4mPHjn3//fe5ubkzZ86USO0Xd3d3fMPwcOCVtbe3V6hVA0DxjaSkhtPZ48ePbW1thSaRNVvGjRtHNrJYLGNjYxEdamlpXbx48fTp0zExMXPmzKmtrR1aP30Ns34OQRB5eXl6enp+fn5nzpyR76o5OTkpyO0vkoVvk1LKVZM9hdrhHUlXP999912EED4VgvX29uLqNEOo2cLj8Y4ePYoQWrJkSXV1NUEQZWVlEqn9cv36dVxZgQxyoDmJAQbTf/HFF/fu3du3b9/YsWMVatUAUHwjaU/N1dV15syZX3/99QcffBAeHn7//v2Kiorm5uacnJz58+fjmi2vX79esGDB77//XlVVlZOTg0SWvjlx4sSaNWtUVFQsLCz09fXff//96dOnD9TPQMrLy9etW7d8+fLo6GgVFZXCwkJNTU1cZh4hVFJSEhgYmJmZicuECcE3rgqW4nn8+PEXX3xx8ODB9evX45KkZDka2a8aACOS/K5R/IWYV0/YbPby5ctNTU3Hjx//2WefRUZGRkRElJSU9PT09FuzRUSZFw6HM23aNB8fnz179kRGRh4/fhwvYqDaLwN59OiRl5eXoaHhzJkzt2zZ8v333wtOFVEY5/Lly/7+/vhTcHNzmzVrlq+v79y5cxkMRm1treCc8lo1QsGuakkWXP2UIIX6nkDpISCKEn8u4pceAm+kUN+TkXT4KUdjxowZaNKJEyfIHS4AFN+///1vExOTN1aRGblG0oUCOWoeGGS0v4ni4uKioqIzZ868/fbbFArFzc1NsEJMa2trYmKirq6upqbm9u3bW1paZBxedna2g4ODnp6eo6Pj+fPnBSeJUxerrq5ux44du3fv/s9//lNXV4cfySzD8CUJ9tSAPDGZTHNzc0XoRLTDhw8jhHCNRi8vL3Nz88rKys2bN+/fvx/PYGBgEB8f39LS0tnZuWPHDqkG01dKSsqVK1eWLl366NGjo0eP+vv7FxcX47vo0JvqYv373/+OjY1tbW3NyMj4xz/+gRAaP348j8eLi4tLTk6W8YpIBCQ1IDetra2hoaHko0nk2IlopaWlV69eJU8YGRsbq6qqdnd3p6SkuLq6BgYGknNaWVnJvsBfR0fHrVu3Lly4gN+GhIS4urp+/vnnOKmJrot1+/ZtX1/fxYsXf/vtt4I1YxwdHS9fvpyenr5u3TrZrIUEweEnkA8+n0+j0fo+GVP2nYjW09PDYDCEdr5sbW0DAgIQQsuXLxcsdqKpqSn7R//evHlz27Zt5FsnJ6f33nuPrDMooi4Wi8Xy8/OzsbH54osv+lbB2rhx486dO0diEWZIakAyCgoKoqKiYmJi5s6dGx8fjx+Rl5OTo6enZ2lpiRBqa2tLTU3V0NDAD9DNy8u7f/8+i8Wi0+n79u178ODBli1b7OzsmpqaFixYYGho6OjoWF1dPahO0FBrPYmQmZnJZrPt7OwEG6lUalZW1jvvvNPW1hYYGDjQs2P63Saia0YRgy8PNWvWLKGi9vr6+hMmTMCvRdTFiouLe/HixdatW/vdidPW1nZwcNi9e/cbA1A48h1RQlKocS6AJObnkpKS4uLiwufzCYJgsVg2NjYeHh74iXCzZ88eN24cOaeDg4OTkxN+7efnN2HCBPw6Li5u9OjRKioqDAajrKysoKDA2NhYS0urqalJ/E6IN9V6EiTmODUfH5/g4GChxqlTpxIE0djYaGRkhBAKDw/H7RkZGfgUu4htIrpm1NAqXwnq7u4eM2ZMZmYm2dJvXayOjg5tbW1NTc0tW7ZMnTp19OjRXl5edXV1gl0lJCTo6+uLKDNDUqjfL+ypgeF6+fJlfHz86tWr1dTUEEJGRkaffvrp9evXT58+jRDS0tISnHmgiudJSUm+vr5UKjU5OdnT03PhwoUZGRlcLjcjI0P8ThBCvr6+7e3tNBpt+OuFPXz4EGeuvqytrfPz89XU1E6ePClUKUDENjEzM5s2bRpCaNeuXXZ2dl5eXjNmzLhz5w5CqKmpKTU1Fd+OoqKiEhQU9Pz586KiokEFXFRUNHbs2PDwcLJFVVX1gw8++Oqrr/Ly8ng83pYtWxBCd+/e5XA477///rJly+7evXvnzp0nT564uroKPo7e1NSUzWY/ePBgUAHIHSQ1MFzV1dUcDmf8+PFki5+fH0KorKxsUP1oaWmpqKjgLIAQCggIUFdXv3fv3mDjGVqtp35xOJwnT54YGBgMNIOnp2daWhpCKDo6+qeffiLbRW+TgWpGkeWh6HQ6nU5vaGgYbOUrPp+/d+/e3NzcfjcCrotVW1uLEMLJi0aj4ftS3nrrrb1793Z0dBw6dIicf/To0QihFy9eiB+AIoCrn2C4Hj9+jBD6888/yRbyyHE43aqqqlpYWMj3aWH4CX64aMJAVq9eXV9ff/jw4eDgYDqdrquri4a6TYZfHiouLi4pKcnGxmagGci6WHg8uWDu8/T0xDGQLfhhZrK/9DFMsKcGhsva2hoh1PcS5DDrLyGEuFzu8DsZDn19fQ0NDVygWBDx14GpaWlpnp6ejY2Nu3btwi1D2yZkeSjBRhaLJWa0hw4dcnd39/DwED0bjgH/K5hk9fT01NTUBHdLcVJWkEfrig+SGhguZ2dnPT29wsJCsuXp06dcLnf+/PkIIVVV1Y6ODnJnp6Ojg6zFRKVScaGRfjGZzObmZlzdZFCdiKj1NFj4OdNCu1cEQQhd7lRVVc3Pz7e2tsZlV9CbtslAhlMeKjs7W0NDY8GCBWRLv5VKybpY5ubmnp6eJSUl5KSWlpauri7BArYsFsvMzMzQ0FCcABQHJDUwXEZGRsnJyZWVleQI2LS0tPDw8JkzZyKEpkyZ8urVq6SkpF9//TUxMZHH4/3yyy93795FCFlYWDx//ry2tvbatWu4aBKPx6urq8OdJCYmhoeH44cci99JSUmJgYFBfn6+pNaORqNVVVUJ7po9e/aMyWQKZVIjI6Nz587p6OiIs00GqhlFlpkKDAz85ptvtm/fvmTJEpyD1q5d6+bmNtBTbi9cuPDll192dXUdOXLkyJEjGRkZUVFR9fX15eXl7777bmpqKv7/QKgu1t69e2/fvk2O2j19+rS9vf2yZcvIbquqqubOnTuszScXcr32+j8KdUkYkMT/XM6ePevj4xMdHb1169Z9+/bh8RwEQbDZbH9/fx0dHScnp5qammXLloWGhp47d44giLq6OktLS1tb27y8PIIgVq5cOWrUKAaDERwcvGLFioSEhCF0IqLWkxAxh3Tw+XwbGxuc1wiC+O677/DxXVBQUHl5udDMhYWF6enporeJiJpR3d3dA5WHmjdvHpVKjY2N7RvhrVu3+p72UldXb2lpEV0XiyCI27dv+/v7r1mzZvv27f/617/YbDY5icvlGhoaNjQ0vHETEQr2+4XSQ0AUWX4udDo9KytroIGsEid+6aHbt28nJCScPXtWBlGJUF5eXlVVJXhwKlVbt27V19ePiYkRZ2aF+v3C4ScAb+Dg4ECj0Y4fPy7HGNrb24uKivAd9TJw8eLFrq4uMTOaooGkBhRFR0cHHkIh70D6ERISYmVldenSJXkFUF9fv3PnTj09PRksq66ujs1m79mzRwbLkgYYpwYUwqlTp65cudLT07Np06ZFixbh6wMKxdvbW45Ld3V1ldmy7O3t7e3tZbY4iYOkBhRCWFhYWFiYvKMAygAOPwEASgWSGgBAqUBSAwAoFUhqAAClAkkNAKBUFOjqZ35+ft9C6UARKPHnosSrJmO49IAiUJTbpG7cuPHkyRN5RwGkJSUlBSHEYDDkHQiQFktLS/zgCLlTlKQGlNtHH32EEMrNzZV3IED5wTk1AIBSgaQGAFAqkNQAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNQAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNQAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUCiQ1AIBSgaQGAFAqkNQAAEoFkhoAQKlAUgMAKBVIagAApQJJDQCgVCCpAQCUiqq8AwDKifX/27vzsKautAHghxCUzYRVwYKoM6CDVbRFDEIRHrYRQRgWRUQWNW7V6WCdwoxbFRR3GFoVF2xHEZVFQcYNUHQoSJVWghXpIm4IKqEYCLEQ4Hx/nG9u0wAhQDbS9/eHDzm5OXnvTfJ67z3nvpfLbWlpoR62tbUhhGpra6kWBoNhYmKihMiAutPAGCs7BqCGvvjii6VLl0pY4MSJE9HR0QqLB/x+QFIDcsHj8UxNTYVCYa/PamlpNTY2MplMBUcFfg/gnBqQCyaT6ePjQ6f3cn6DTqfPmzcPMhqQE0hqQF7Cw8O7urp6tnd3d4eHhys+HvA7AYefQF5++eUXExMTMkQgSldXl8vl6ujoKCUqoPZgTw3Ii7a2dmBgoJaWlmijlpZWcHAwZDQgP5DUgByFhYWJjRUIhcKwsDBlxQN+D+DwE8hRZ2fnmDFjfv75Z6rFwMCgsbGx1wEEAGQC9tSAHNHp9EWLFlFHoFpaWuHh4ZDRgFxBUgPytWjRIuoIVCgULlq0SLnxALUHh59AvjDGlpaWL168QAiZm5u/ePFCQ0ND2UEBdQZ7akC+NDQ0lixZMmLEiBEjRkRGRkJGA/IGe2pA7qqqquzs7MgfU6dOVXY4QM2pyinbAwcO3L59W9lRAHnR19dHCG3fvl3ZgQB5cXR0XL9+vbKjQEh1Dj9v375dXl6u7CiAuPLycpl8LlZWVuPHjx96PzJUV1eXnZ2t7CjURHl5uerslKjKnhpCiMViZWVlKTsK8BshISEIoaF/LqSS2sSJE2UQk4xkZmYuXLgQvnIyQb4nKkKFkhpQYyqVzoB6U5XDTwAAkAlIagAAtQJJDQCgViCpAfD78vjx455F7tQJJDUgY7m5uZaWlg8fPlR2IDJWUFCQn59/9uzZKVOmaGhoODs7d3Z2Us82NzcnJCSMGjVKR0dn69atTU1NCg4vIyPD3t6ewWA4ODhcunRJ9KnW1lYDAwON/wkMDNTT0xN7OYfD2bZt286dO589e8bhcD7//PPhOy0fRj+BjOnp6Y0ePVpbW1t+b9HQ0GBubi6//ns6fPgwQmj16tUIIQ8PD3Nz89LS0k8++eTAgQNkAUNDw02bNjU1Nb19+3bbtm2KjA0hlJSUVFhYuGTJkidPnhw9etTPz6+goMDDw4M8m5aWFhQURA1Ae3l5ib728ePHsbGxzc3Nqampf/jDHxBC48aNa29vj4uL2717t4JXRCYgqQEZ8/T09PT0lF//zc3N4eHh169fl99biLl+/fqNGzeoGW0mJiZ0Or2zszMpKcnJySkoKIha0srKSnT3TTH4fP6dO3cuX75MHi5cuNDJyWnv3r0kqXV1deXl5RUWFvZa8amiosLHx2fRokXnzp0TvSzXwcHh2rVrBw8e/PDDDxWzFjIEh59gOOno6AgLCxO9KbK8dXV1xcTEiO182djY+Pv7I4SWLl36448/Uu06OjqKr1T+9ddfb9myhXrIYrFmzJjx008/kYc5OTmVlZWhoaFHjx4Vvb00QojL5fr6+lpbW+/fv79noYH169dv37790aNH8o5f5iCpAVlqbm5OS0vz9PTMzc1FCFVWVv7973+fOHFic3NzVFSUiYmJg4MDSUnV1dUbN260tbWtr68PCAgwMjJycHAgl2SdOXOGwWBYWloihFpaWpKTk7W1tR0dHRFCWVlZDx484HK5bDZ73759CKHS0lJLS8srV67IaY3S0tJ4PJ6tra1oI41GS09Pf/fdd1taWoKCgt6+fdvra3NyctauXbthw4a5c+du2rSpvb1d8jZBCGGMU1NTV69ePWvWLC8vL9GM2Rd3d/c//elPoi1MJpO6KK24uFggEOTk5KxcudLW1ragoIBaLC4u7tWrV5s3b+51J05PT8/e3n7nzp39BqBysGoIDg4ODg5WdhRA3EA/l+rq6piYGIRQdnY2xrihoYEcBK1cufLBgweFhYUMBiM0NBRjHBcXZ2BgoKmpGRMTU1xcnJOTY2JioqurW19fjzH28vKysLCgurW3t2exWORvX1/f8ePHU09dunRJR0fn9OnTA121c+fOSfP99/b2DgkJEWucPn06xri2ttbY2BghFBkZSdpTU1PJKXaMcVJS0uzZszs6OjDGXC7X2tp6zpw53d3dErYJxjgxMfHLL7/EGHd2drJYLDMzs7a2tgGtV2dnp6mpaVpaGtUiFAorKiqioqJoNJq2tnZ1dTXGmM/n6+np6ejobNy4cfr06QYGBh4eHhwOR7Sr+Ph4JpPZ2dnZ75uq1O8XkhqQZBCfy82bN6mkhjH+xz/+gRDicrnk4bx586ytrcnfYWFhWlpa5GePMSaXl2/ZsgVjHBAQIJrUWCxWX0kNYyzNr64nKZPauHHjVq1aJdZIkhrGuLi4mBQrP3r0KBZJaq9evdLT0zt58iT1ki+++AIhdOrUKdz3Nnnx4sWYMWO6urpIO9kVPXv27IDW68KFC9OnT+91m+Tk5GhoaPzlL3/BGJeUlCCEnJycfvzxR4zxo0ePJk2apK+v/+LFC2r5o0ePIoSqqqr6fVOV+v3C4SeQMbFjGU1NTdHGUaNGtba2kr91dXU1NTWpOxj4+/uPHDny/v37A31H8hby0NbW9vz5c0NDw74WcHV1TUlJQQitW7fu22+/pdrLy8vb2trGjRtHtfj6+iKEiouLUd/bpKysTCgUrly5ks1ms9nsmpqa5cuXD+gkXUdHx549ezIzM3vdJoGBgSEhIZWVlQih+vp6hFBYWNgf//hHhNDEiRP37NnD5/MPHTpELW9gYIAQevXqlfQBqAIY/QSqgk6njx07VvGjhxIIhUKMca/3maesWrWqqqrq8OHDISEhbDZ71KhRCKGnT58ihERvo0UdXEvo6uHDh3p6eseOHRt0wHFxcYmJidbW1n0t4OLiQvbRTE1N0W//P3B1dSUxUC00Gg0hNOxu0gp7akCFCASCyZMnKzuKXzGZTG1t7Tdv3oi1499OTE1JSXF1da2trd2xYwdpmTBhAvpfwSVRktdOV1e3rq6urq5OtJHL5UoZ7aFDh1xcXObMmSN5MRID+Vc0yTIYDC0tLdHdUpKUxUYhVB8kNaAqGhoaGhsbg4ODEUJ0Op3P51O7SHw+v7u7m/xNo9HEbpBMPSVzGhoas2fPFtu9whiLDXfS6fTs7OwJEybw+XzS4ujoyGAwyBAwUVdXJxAI5s+fL+Htpk6dijGOjY2lWl6/fk1OxvUrIyNDW1s7ICCAaiF7ZGJu3boVHR2NEDI3N3d1dS0qKqKeampqEgqFLBaLauFyuWZmZkZGRtIEoDogqQEZIz94Mn0BIUQSEHVQ+fbtW4FAQC3c3t7O4XDI3wkJCZGRkQ4ODgihqVOnvnnzJjEx8YcffkhISGhvb//+++/v3buHEBo7duzLly8rKytv3rwpEAiKiooMDQ3lV8M2LCysrKxMdNfsxYsXDQ0NYonV2Nj44sWLpGo5ebh79+7S0lJqknBKSkpkZKSbmxvqe5t4enrOnDkzIyMjKCjo1KlTW7duXbx4MclBa9ascXZ2pmafibl8+fJnn30mFAqPHDly5MiR1NTUtWvXVlVVlZSUTJs2LTk5mfz3kJubq6Ojs2TJEvKqPXv2VFRUULN2T58+bWdnFxUVRXVbVlY2d+7cIW0+pVDqMMWvVGr0BFAG+rncvn3bx8cHIeTi4lJaWlpUVETOQ69Zs+b169cnT55kMpkIoU8//bSzs3P58uUjRoyIiYkJCQlZtmxZfHx8d3c36YfH4/n5+enr67NYrLt370ZFRYWHh1+8eBFjzOFwLC0tbWxssrKyMMY3btwwNzfPzc0d6KpJOfrZ0dFhbW1N8hrG+Pz58+T4Ljg4uKSkRGzh3NzcgwcPUg/z8vK8vb3XrVu3efPmffv2kbWTvE2ampoWL148evRoU1PTiIgIaixy3rx5NBotNja2Z4R37tzpedpr5MiRTU1NT5488fDwMDIycnNz27hx44ULF8ReW1FR4efnt3r16q1bt/71r3/l8XjUUwKBwMjIqKampv9NqWK/X1W5m5SsykYD2ZLr58Jms9PT0/uauSpvpJy3NN//ioqK+Pj4vLw8BUQlQUlJSVlZmejBqVxt3ryZyWRu2LBBmoVV6vcLh58A9MPe3j4sLOz48eNKjKG1tTU/P59cUa8AV65cEQqFUmY0VTPskxp1ahYMO3w+n8yZUHYg/Vu4cKGVldXVq1eVFUBVVdX27dsZDIYC3ovD4fB4vF27dingveRhGM9TO3bsWGZmZk1NzfPnz5Udy684HE5ubq6WllZ4eLjo3MueLl++fPjw4f/85z8IIUdHRxqN1tbWNnLkyDlz5qxYsYIUgVFjJ0+eLCws7Orq+vjjj0NDQ8n4gCqTa+mRfjk5OSnsvezs7MjNp4epYZzUli5devLkSdWZq9mzLpVkPj4+dnZ2FhYWVlZWZWVlpPHu3btbtmyZNGlSbGxsfHw8mf2oliIiIiIiIpQdBVBDw/g3o6mpaWFhoewo/l9FRcWsWbPMzc0LCgqk38kiBUhFh65mzpx56dKl0NDQnTt3DtMSfQAo1zBOaqpDcl0qCXpdmEajHTx4cPTo0QkJCc+ePZNdmAD8Lgy/pJaXl7dixYrY2Nh169Y1NDRQ7bi3QlSSa1dVVlZGR0fv3r3b39+fOmPSaz+SSahLNbhqX0wmc8GCBQKBIDMzU7mrBsDwo8Q5cqKknLx3+vTpWbNmvX37FmPc2NhoampqZmZGnuq1EJXk2lWTJk366quvMMbt7e2+vr4S+pEQkuS6VJKrfZGLCidPntzzqfT0dIRQdHS0ElcNq9ikStmScvItkIZKfU9U5UOVZqO0tbWZm5tnZGRQLYGBgSSpSShE1Vftqo6ODg0NjX/961+k/erVq5L76Uu/dakkVPuSkNSuXbuGEHJ3d1fiqmEV+7LKFiQ1GVKp78lwGv0sKSlpaGiYOnUq1TJixAjyB1WIinqKKkTVV+0qLS0tLy+vv/3tb999992uXbu8vb0l99OXXutS+fv7Hzp0KCEhAQ222hePx0MI2djYKHHViOzs7AGdKBxe1HjVFIxUIlAFwymp1dTUIJFEJmpwhajOnj27aNGiY8eOXbhwITMz083NbRD9SFOXahDIytrZ2Slx1QgWi0UqdKuZ27dvJycnk/01MERJSUnKDuFXwympkXT29OlTGxsbsaeoQlSikzy4XK6JiYmEDnV1da9cuXL69OkNGzb8+c9/rqysHEQ/0tSlGiiMcVZWFoPB8PX1PXv2rLJWjbCwsFiwYMGg10WVJScnq+uqKZiKXPVJDKfRz2nTpiGERP9r7e7uJjVVBlGIqr29nZRgX7x4cXl5Oca4uLh4EP30W5dKQrUv3McVQvv3779///6+ffveeecdJa4aAMPRcNpTc3JycnNz+/LLL99///3IyMgHDx589dVXjY2NZ86cmT9/PilE9csvvwQEBPz0009lZWVnzpxBEut5nThxYvXq1ZqammPHjmUyme+9996sWbP66keCPXv2ODs7X758mVTdEa1LVVRUFBQUlJaW1usZB3Lhqmh9sadPn+7fv//zzz//6KOP2Gw2EqmxpZRVA2D4Ud4YxW9IOXrC4/GWLl06ZsyYcePGffrppytWrIiOji4qKurq6uq1EJWE2lVtbW0zZ8709vbetWvXihUrjh8/Tt6ir4JWkvVVl0pCta9r1675+fmRT8HZ2dnd3d3Hx2fu3LkxMTGVlZWiSypx1VRqVEu2YPRThlTqewL11IAkavy5SF9PDfRLpb4nw+nwU4nIEGevTpw4Qe1wAaD6Hj9+PHr0aHLdsVoaTgMFStTYN8hovxMFBQX5+flnz56dMmWKhoaGs7OzaIWY5ubmhISEUaNG6ejobN26tampScHhZWRk2NvbMxgMBweHS5cuiT7V2tpqYGCg8T+BgYE9MxqHw9m2bdvOnTufPXvG4XDILZkVGL4swZ4aUKaGhgZzc3N0WMqKAAAgAElEQVRV6ESyw4cPI4RI4VkPDw9zc/PS0tJPPvnkwIEDZAFDQ8NNmzY1NTW9fft227Ztcg2mp6SkpMLCwiVLljx58uTo0aN+fn4FBQXkKjqEUFpaWlBQ0MSJE8lDLy8v0df2LJk1bty49vb2uLi4YVonBpIaUJrm5ubw8HDqfktK7ESy69ev37hxgzphZGJiQqfTOzs7k5KSnJycgoKCqCWtrKwUX+CPz+ffuXOHuinUwoULnZyc9u7dS5JaV1dXXl5eYWFhz2oLCKGKigofH59FixadO3dO9OIKBweHa9euHTx48MMPP1TMWsgQHH4C5ejo6AgLC+t5u1/FdyJZV1dXTEyM2M6XjY2Nv78/Qmjp0qWixU50dHQUfz/zr7/+esuWLdRDFos1Y8YM6mZ6OTk5lZWVoaGhR48ebWlpEX2h5JJZ69ev3759+6NHj+Qdv8xBUgOykZOTs3bt2g0bNsydO3fTpk3kvp9nzpxhMBiWlpYIoZaWluTkZG1tbUdHR4RQVlbWgwcPuFwum83et29fdXX1xo0bbW1t6+vrAwICjIyMHBwcysvLB9QJGmytJwnS0tJ4PJ6tra1oI41GS09Pf/fdd1taWoKCgvq6IVav20RyzSg88PJQ7u7uYjdRZzKZ48ePJ38XFxcLBIKcnJyVK1fa2toWFBRQi0komYUQ0tPTs7e337lzZ78BqBzlziihqNQ8F0CR8nNJSkqaPXt2R0cHxpjL5VpbW8+ZM4fc5tLLy8vCwoJa0t7ensVikb99fX3Hjx9P/o6LizMwMNDU1IyJiSkuLs7JyTExMdHV1a2vr5e+E9xfrSdRUs5T8/b2DgkJEWucPn06xri2ttbY2BghFBkZSdpTU1PJKXYJ20RyzahBlIcS09nZaWpqmpaWRrUIhcKKioqoqCgajaatrV1dXY37K5lFxMfHM5lMCWVmKCr1+4U9NTBUr1+/3rRp06pVq7S0tBBCxsbG//znP2/dunX69GmEkK6urujCve4UIIQSExN9fHxoNNru3btdXV0DAwNTU1MFAkFqaqr0nSCEfHx8Wltbw8LChr5exMOHD0nm6mnChAnZ2dlaWlr//ve/xSoFSNgmZmZmM2fORAjt2LHD1tbWw8Pjgw8++OabbxBC9fX1ycnJ5A7qmpqawcHBL1++zM/PH1DA+fn577zzTmRkJNVCp9Pff//9L774Iisrq729fePGjQihe/futbW1vffee1FRUffu3fvmm2+eP3/u5OQkehXzmDFjeDxedXX1gAJQOkhqYKjKy8vb2tpEb53l6+uLECouLh5QP7q6upqamiQLIIT8/f1Hjhx5//79gcYzuFpPvWpra3v+/LmE2gSurq4pKSkIoXXr1n377bdUu+Rt0lfNKKo8FJvNZrPZNTU1UpaHonR0dOzZsyczM7PXjRAYGBgSElJZWYn6KJnF5/MPHTpELW9gYIAQevXqlfQBqAIY/QRD9fTpU4TQzz//TLVQR45D6ZZOp48dO1a5dwsjtyUlRRP6smrVqqqqqsOHD4eEhLDZ7FGjRqHBbpNBl4eixMXFJSYmWltb97WAi4sLKWsqTckscjMzxQ99DBHsqYGhmjBhAkKo5xAkKco0FAKBYOidDAWTydTW1iYFikXh305MTUlJcXV1ra2t3bFjB2kZ3DahykOJNnK5XCmjPXTokIuLy5w5cyQvRmKQpmQWScpioxCqD5IaGCpHR0cGg5Gbm0u11NXVCQSC+fPnI4TodDqfz6d2dvh8PlWLiUajkUIjvWpoaGhsbCTVTQbUiYRaTwOloaExe/Zssd0rjLHYcCedTs/Ozp4wYQIpu4L62yZ9GUp5qIyMDG1t7YCAAKqF7JGJuXXrVnR0NJKiZBZCiMvlmpmZGRkZSROA6oCkBobK2Nh49+7dpaWl1AzYlJSUyMhINzc3hNDUqVPfvHmTmJj4ww8/JCQktLe3f//99/fu3UMIjR079uXLl5WVlTdv3iRFk9rb2zkcDukkISEhMjKS3Lld+k6KiooMDQ2zs7NltXZhYWFlZWWiu2YvXrxoaGgQy6TGxsYXL17U19eXZpv0VTOKKjMVFBR06tSprVu3Ll68mOSgNWvWODs7U7PPxFy+fPmzzz4TCoVHjhw5cuRIamrq2rVrq6qqSkpKpk2blpycTP4/yM3N1dHRIQMRCKE9e/ZUVFRQs3ZFS2YRZWVlc+fOHdLmUwqljr3+SqWGhAFF+s8lLy/P29t73bp1mzdv3rdvH5nPgTHm8Xh+fn76+vosFuvu3btRUVHh4eEXL17EGHM4HEtLSxsbm6ysLIzx8uXLR4wYERMTExISsmzZsvj4+EF0IqHWkxgpp3R0dHRYW1uTvIYxPn/+PDm+Cw4OLikpEVs4Nzf34MGDkreJhJpRnZ2dfZWHmjdvHo1Gi42N7RnhnTt3ep72GjlyZFNT05MnTzw8PIyMjNzc3DZu3HjhwgWx1/ZVMgtjLBAIjIyMampq+t1EWMV+v1B6CEiiyM+FzWanp6f3NZFV5qQvPVRRUREfH5+Xl6eAqCQoKSkpKysTPTiVq82bNzOZzA0bNkizsEr9fuHwE4B+2Nvbh4WFHT9+XIkxtLa25ufnkyvqFeDKlStCoVDKjKZqIKkBVcHn88kUCmUH0ouFCxdaWVldvXpVWQFUVVVt376dwWAo4L04HA6Px9u1a5cC3kseYJ4aUAknT54sLCzs6ur6+OOPQ0NDyfiASvH09FTiuzs5OSnsvezs7Ozs7BT2djIHSQ2ohIiIiIiICGVHAdQBHH4CANQKJDUAgFqBpAYAUCuQ1AAAakWFBgrq6uoyMzOVHQX4DXJxtVp+Lrdv30ZqumqKV1dXZ2Fhoewo/kfJVzT8D7luGQAwTMFlUuD3ZcGCBQh2i4BCwDk1AIBagaQGAFArkNQAAGoFkhoAQK1AUgMAqBVIagAAtQJJDQCgViCpAQDUCiQ1AIBagaQGAFArkNQAAGoFkhoAQK1AUgMAqBVIagAAtQJJDQCgViCpAQDUCiQ1AIBagaQGAFArkNQAAGoFkhoAQK1AUgMAqBVIagAAtQJJDQCgViCpAQDUCiQ1AIBagaQGAFArkNQAAGoFkhoAQK1AUgMAqBVIagAAtQJJDQCgViCpAQDUCiQ1AIBaoSs7AKCe/vvf/96+fZt6WFNTgxDavXs31eLo6Oji4qKEyIC608AYKzsGoIauX7/u4eGhpaVFo4kfDXR3dwuFwqKiInd3d6XEBtQbJDUgF93d3WZmZo2Njb0+a2Ji8vLlS01NTQVHBX4P4JwakAsajbZ48eIRI0b0fGrEiBHh4eGQ0YCcQFID8rJo0aKOjo6e7R0dHYsWLVJ8POB3Ag4/gRyNHz/+6dOnYo2WlpZPnz7V0NBQSkhA7cGeGpCjJUuWaGlpibZoaWlFRUVBRgPyA3tqQI5qamr+9Kc/iTV+9913U6ZMUUo84PcA9tSAHE2ePHnKlCmi+2W2traQ0YBcQVID8hUREUENdGppaUVGRio3HqD24PATyNfz58+trKzI10xDQ6O2tnb8+PHKDgqoM9hTA/JlaWk5a9YsGo1Go9FmzZoFGQ3IGyQ1IHdLlizR0NCg0WhLlixRdixA/cHhJ5A7LpdrZmaGEKqvrx89erSywwHqDquG4OBgZW8JAMDgBQcHKzuL/D8VKj3EYrFiYmKUHQX4jaSkJITQ0D+X//73vxoaGh988IEsgpKN27dvJycnnzt3TtmBqAPyPVERKpTULCwsFixYoOwowG9kZWUhhIb+ucydOxchNGrUKBnEJDvJycnwlZMJ8j1RESqU1IAaU7V0BtQYjH4CANQKJDUAgFqBpAYAUCuQ1AD4fXn8+HFbW5uyo5AjSGpAxnJzcy0tLR8+fKjsQGSsoKAgPz//7NmzpO6Is7NzZ2cn9Wxzc3NCQsKoUaN0dHS2bt3a1NSk4PAyMjLs7e0ZDIaDg8OlS5dEn2ptbTUwMND4n8DAQD09PbGXczicbdu27dy589mzZxwO5/PPP8fDdlo+jH4CGdPT0xs9erS2trb83qKhocHc3Fx+/fd0+PBhhNDq1asRQh4eHubm5qWlpZ988smBAwfIAoaGhps2bWpqanr79u22bdsUGRtCKCkpqbCwcMmSJU+ePDl69Kifn19BQYGHhwd5Ni0tLSgoaOLEieShl5eX6GsfP34cGxvb3Nycmpr6hz/8ASE0bty49vb2uLg40VsaDiOQ1ICMeXp6enp6yq//5ubm8PDw69evy+8txFy/fv3GjRvUVCwTExM6nd7Z2ZmUlOTk5BQUFEQtaWVlJbr7phh8Pv/OnTuXL18mDxcuXOjk5LR3716S1Lq6uvLy8goLC+n0Xn7sFRUVPj4+ixYtOnfunGjZOwcHh2vXrh08ePDDDz9UzFrIEBx+guGko6MjLCystrZWYe/Y1dUVExMjtvNlY2Pj7++PEFq6dOmPP/5Itevo6Ojo6CgsNuLrr7/esmUL9ZDFYs2YMeOnn34iD3NyciorK0NDQ48ePdrS0iL6Qi6X6+vra21tvX///p4F1tevX799+/ZHjx7JO36Zg6QGZKm5uTktLc3T0zM3NxchVFlZ+fe//33ixInNzc1RUVEmJiYODg4kJVVXV2/cuNHW1ra+vj4gIMDIyMjBwaG8vBwhdObMGQaDYWlpiRBqaWlJTk7W1tZ2dHRECGVlZT148IDL5bLZ7H379iGESktLLS0tr1y5Iqc1SktL4/F4tra2oo00Gi09Pf3dd99taWkJCgp6+/Ztr6/NyclZu3bthg0b5s6du2nTpvb2dsnbBCGEMU5NTV29evWsWbO8vLxEM2Zf3N3dxWqmM5lMqsRTcXGxQCDIyclZuXKlra1tQUEBtVhcXNyrV682b97c606cnp6evb39zp07+w1A5Sj74tP/FxwcrDoXxALKQD+X6upqcqFodnY2xrihoYEcBK1cufLBgweFhYUMBiM0NBRjHBcXZ2BgoKmpGRMTU1xcnJOTY2JioqurW19fjzH28vKysLCgurW3t2exWORvX1/f8ePHU09dunRJR0fn9OnTA101ctVnv4t5e3uHhISINU6fPh1jXFtba2xsjBCKjIwk7ampqeQUO8Y4KSlp9uzZHR0dGGMul2ttbT1nzpzu7m4J2wRjnJiY+OWXX2KMOzs7WSyWmZlZW1vbgNars7PT1NQ0LS2NahEKhRUVFVFRUTQaTVtbu7q6GmPM5/P19PR0dHQ2btw4ffp0AwMDDw8PDocj2lV8fDyTyezs7Oz3TVXq9wtJDUgyiM/l5s2bVFLDGP/jH/9ACHG5XPJw3rx51tbW5O+wsDAtLS3ys8cYZ2dnI4S2bNmCMQ4ICBBNaiwWq6+khjGW5lfXk5RJbdy4catWrRJrJEkNY1xcXExul3X06FEsktRevXqlp6d38uRJ6iVffPEFQujUqVO4723y4sWLMWPGdHV1kXayK3r27NkBrdeFCxemT5/e6zbJycnR0ND4y1/+gjEuKSlBCDk5Of34448Y40ePHk2aNElfX//FixfU8kePHkUIVVVV9fumKvX7hcNPIGNixzLkBgVU46hRo1pbW8nfurq6mpqa1D30/P39R44cef/+/YG+o/xu9t7W1vb8+XNDQ8O+FnB1dU1JSUEIrVu37ttvv6Xay8vL29raxo0bR7X4+voihIqLi1Hf26SsrEwoFK5cuZLNZrPZ7JqamuXLlw/oJF1HR8eePXsyMzN73SaBgYEhISGVlZUIofr6eoRQWFjYH//4R4TQxIkT9+zZw+fzDx06RC1vYGCAEHr16pX0AagCGP0EqoJOp48dO1bxo4cSCIVCjHFXV5eEZVatWlVVVXX48OGQkBA2m00u3Se3cP7555+pxaiDawldPXz4UE9P79ixY4MOOC4uLjEx0drauq8FXFxcyD6aqakp+u3/B66uriQGqoVGoyGEFD/0MUSwpwZUiEAgmDx5srKj+BWTydTW1n7z5o1YO/7txNSUlBRXV9fa2todO3aQlgkTJiCEeo7SSl47XV3durq6uro60UYulytltIcOHXJxcZkzZ47kxUgM5F/RJMtgMLS0tER3S0lS7nnnVhUHSQ2oioaGhsbGRlIDmU6n8/l8aheJz+d3d3eTv2k0mlAoFH0h9ZTMaWhozJ49W2z3CmMsNtxJp9Ozs7MnTJjA5/NJi6OjI4PBIEPARF1dnUAgmD9/voS3mzp1KsY4NjaWann9+jU5GdevjIwMbW3tgIAAqoXskYm5detWdHQ0Qsjc3NzV1bWoqIh6qqmpSSgUslgsqoXUYTcyMpImANUBSQ3IGPnBk+kLCCGSgKiDyrdv3woEAmrh9vZ2DodD/k5ISIiMjHRwcEAITZ069c2bN4mJiT/88ENCQkJ7e/v3339/7949hNDYsWNfvnxZWVl58+ZNgUBQVFRkaGhIBhnkISwsrKysTHTX7MWLFw0NDWKJ1djY+OLFi/r6+tTD3bt3l5aWUpOEU1JSIiMj3dzcUN/bxNPTc+bMmRkZGUFBQadOndq6devixYtJDlqzZo2zszM1+0zM5cuXP/vsM6FQeOTIkSNHjqSmpq5du7aqqqqkpGTatGnJycnkv4fc3FwdHR3q9jd79uypqKigZu2ePn3azs4uKiqK6rasrIxU9xxmlDpM8SuVGj0BlIF+Lrdv3/bx8UEIubi4lJaWFhUVkfPQa9asef369cmTJ5lMJkLo008/7ezsXL58+YgRI2JiYkJCQpYtWxYfH9/d3U364fF4fn5++vr6LBbr7t27UVFR4eHhFy9exBhzOBxLS0sbG5usrCyM8Y0bN8zNzXNzcwe6alKOfnZ0dFhbW5O8hjE+f/48Ob4LDg4uKSkRWzg3N/fgwYPUw7y8PG9v73Xr1m3evHnfvn1k7SRvk6ampsWLF48ePdrU1DQiIoIai5w3bx6NRouNje0Z4Z07d3qe9ho5cmRTU9OTJ088PDyMjIzc3Nw2btx44cIFsddWVFT4+fmtXr1669atf/3rX3k8HvWUQCAwMjKqqanpf1Oq2O9XVe4mFRISglSsKDBAcv5c2Gx2enp6XzNX5S0zM3PhwoXSfP8rKiri4+Pz8vIUEJUEJSUlZWVlogencrV582Ymk7lhwwZpFlap3++wP/ykzmIAICf29vZhYWHHjx9XYgytra35+fnkinoFuHLlilAolDKjqZphnNSOHTvm6empCkMzXl5eGj3k5+dLftXly5f9/PzIwrNnz3Z2dp4xYwaLxYqNjR2OF9wNAp/PJ3MmlB1I/xYuXGhlZXX16lVlBVBVVbV9+3YGg6GA9+JwODweb9euXQp4L3kYxvPUli5devLkSaVPa3r8+HF9fX1CQgK5Xy9C6NmzZ3v37nV3d5f8Qh8fHzs7OwsLCysrq7KyMtJ49+7dLVu2TJo0KTY2Nj4+nkwUUksnT54sLCzs6ur6+OOPQ0NDyfiAKpNr6ZF+OTk5Key97Ozs7OzsFPZ2MjeMk5qmpqaFhUVf40EKc+PGjeLiYjKVkThw4ICXl5eurm6/ryW1+kTP8s6cOfPSpUsRERE7d+7U19cn19OopYiIiIiICGVHAdSQ2u4IKMyyZctEMxpC6Pz586LThSToWe8FIUSj0Q4ePDh69OiEhIRnz57JJkoAfjeGX1LLy8tbsWJFbGzsunXrGhoaqHbcW80WyWVeKisro6Ojd+/e7e/vTx1c9NqP9F6+fHnnzh1yoR8abGEcJpO5YMECgUCQmZmpOqsGwPCgxOkkoqSc53L69OlZs2a9ffsWY9zY2GhqampmZkae6rVmi+QyL5MmTfrqq68wxu3t7b6+vhL6kX5FUlNTXVxcqIeSC+OQ628mT57c86n09HSEUHR0tHJXTaXmH8mWlPPUgDRU6nuiKh+qNBulra3N3Nw8IyODagkMDCRJTULNlr7KvHR0dGhoaPzrX/8i7VevXpXcj5S8vLwOHDgg2iKhMI6EpHbt2jWEkLu7u3JXTaW+rLIFSU2GVOp7MpwGCkpKShoaGqZOnUq1jBgxgvxB1WyhnqJqtvRV5kVLS8vLy+tvf/vbd999t2vXLm9vb8n9SOPNmzfFxcXkJh2UwRXG4fF4CCEbGxulr1pdXR05ClYzt2/fRgip5aopXl1dnYWFhbKj+H/DKanV1NQgkUQmanA1W86ePbto0aJjx45duHAhMzPTzc1tiLVf/vOf/0yePJm6bc9QkJW1s7NT+qqVl5cvXLhwoK8aLtR41RSMVCJQBcNpoICkM1KpSszgarbo6upeuXIlPT2dTqf/+c9/fvjw4RBrv0g/7ikZxjgrK4vBYPj6+ip91VTnsEK24PBThlQno6HhldSmTZuGECLfRaK7u5uUHxhEzZb29nZSrXjx4sXl5eUY4+Li4qHUfhEIBNeuXeuZ1CQUxsF9TKbfv3///fv39+3b984776jCqgEwjAynw08nJyc3N7cvv/zy/fffj4yMfPDgwVdffdXY2HjmzJn58+eTmi2//PJLQEDATz/9VFZWdubMGSSx9M2JEydWr16tqak5duxYJpP53nvvzZo1q69++nXt2jVjY+P33ntPtLGoqCgoKCgtLa3X/8rIhauipXiePn26f//+zz///KOPPmKz2UikHI0SVw2A4USJu6yipBw94fF4S5cuHTNmzLhx4z799NMVK1ZER0cXFRV1dXX1WrNFQpmXtra2mTNnent779q1a8WKFcePHydv0Vftl36Fh4evXbtWrFFCYZxr1675+fmRT8HZ2dnd3d3Hx2fu3LkxMTGVlZWiSypx1VRqVEu24PBThlTqewKlh4Akavy5SF96CPRLpb4nw+nwU4nELoQSdeLECWqHCwCgdJDUpNLY2KjsEAAAUhlOo58AqKaCgoL8/PyzZ89OmTJFQ0PD2dlZtCJWc3NzQkLCqFGjdHR0tm7d2tTUpODwMjIy7O3tGQyGg4PDpUuXSCOHwyH3XVZwMAoASQ0ok2hJAuV2MmiHDx9+9OiRn59faGjorVu36HR6aWnpJ598Qi1gaGi4adOm5cuXR0ZGbtu2zdjYWJHhJSUlpaenL1myZNmyZQ8ePPDz8yN3kLKzs3NwcIiLi1NkMIoBSQ0oTXNzc3h4uCp0MmjXr1+/ceMGVWXbxMSEXLWWlJSUk5MjuqSVlRUZrVYkPp9/586dy5cvf/TRR0lJSdevX9fQ0Ni7dy951sHBQV9f/+DBgwqOSt4gqQHl6OjoCAsL63m7X8V3MmhdXV0xMTHbtm0TbbSxsfH390cILV26VLS4k46OjuJvdf71119v2bKFeshisWbMmCFaV3X9+vXbt29Xs/LxkNSAbOTk5Kxdu3bDhg1z587dtGkTue/nmTNnGAyGpaUlQqilpSU5OVlbW9vR0REhlJWV9eDBAy6Xy2az9+3bV11dvXHjRltb2/r6+oCAACMjIwcHh/Ly8gF1ggZbwG5w0tLSeDyera2taCONRktPT3/33XdbWlqCgoL6uldWr5tLco08PPByeO7u7mI38WAymePHj6ce6unp2dvb79y5c4CrrtqUO02OolKT9wBFys8lKSlp9uzZHR0dGGMul2ttbT1nzhxym0svLy8LCwtqSXt7exaLRf729fUdP348+TsuLs7AwEBTUzMmJqa4uDgnJ8fExERXV7e+vl76TnB/BexEDX3yrbe3d0hIiFjj9OnTMca1tbXk3FlkZCRpT01NJSfmcd+bS3KNvCFW+iMvNDU1TUtLE22Mj49nMpkSCmRJQ6V+v7CnBobq9evXmzZtWrVqlZaWFkLI2Nj4n//8561bt06fPo0QErtXA1UoSUxiYqKPjw+NRtu9e7erq2tgYGBqaqpAIEhNTZW+E4SQj49Pa2trWFjY0NerXw8fPuzrrP+ECROys7O1tLT+/e9/i1VGkbC5zMzMZs6ciRDasWOHra2th4fHBx988M033yCE6uvrk5OTyc3VNTU1g4ODX7582e8dy8Tk5+e/8847kZGRoo1jxozh8XjV1dUD6kqVQVIDQ1VeXt7W1jZu3DiqhVQzLy4uHlA/urq6mpqa5KeOEPL39x85cuT9+/cHGs/gCtgNVFtb2/Pnzw0NDftawNXVNSUlBSG0bt26b7/9lmqXvLn6qpFHlcNjs9lsNrumpmZAlf4QQh0dHXv27MnMzBTbPgYGBgihV69eSd+VioPJt2CoSDGon3/+mWqhjhyH0i2dTh87dqzSb4HYF3LHUlIkpi+rVq2qqqo6fPhwSEgIm80eNWoUGuzmGmKlP4RQXFxcYmKitbW1WDu5DaPiBzHkB/bUwFBNmDABIdRzCHLy5MlD7FkgEAy9EzlhMpna2tqkILso/NvprCkpKa6urrW1tTt27CAtg9tcQ6z0d+jQIRcXlzlz5vR8iqRXVbgpuKxAUgND5ejoyGAwcnNzqZa6ujqBQDB//nyEEJ1O5/P51B4Nn8+nCszRaDRSPalXDQ0NjY2NpGTTgDqRUMBOhjQ0NGbPni22e4UxFhvupNPp2dnZEyZMIGWmUH+bqy9DKYeXkZGhra0tWumvpKSE+pvL5ZqZmRkZGUnT1bAASQ0MlbGx8e7du0tLS69fv05aUlJSIiMj3dzcEEJTp0598+ZNYmLiDz/8kJCQ0N7e/v3339+7dw8hNHbs2JcvX1ZWVt68eZNUgmtvb+dwOKSThISEyMhIcud26TspKioyNDTMzs5WwIqHhYWVlZWJ7pq9ePGioaFBLMkaGxtfvHhRX1+feihhc/VVI48qqxcUFHTq1KmtW7cuXrw4OjoaIbRmzRpnZ+e+7up9+fLlzz77TCgUHjly5MiRI6mpqWvXrq2qqqIWKCsrmzt3row2iWpQ6tjrr1RqSBhQpP9c8vLyvL29161bt3nz5n379pH5HBhjHo/n5+enr6/PYrHu3r0bFRUVHh5+8eJFjDGHw7G0tLSxscnKysIYL1++fMSIETExMSEhIcuWLYuPjx9EJxIK2N1e88UAAAGnSURBVIkZ+pSOjo4Oa2trktcwxufPnyfHd8HBwSUlJWIL5+bmHjx4UPLmklAjr7Ozs69yePPmzaPRaLGxsT0jvHPnTs+TZSNHjmxqaiILCAQCIyOjmpqaoWwHrGK/X6inBiRR5OfCZrPT09P7mq0qczKpp1ZRUREfH5+XlyerqAanpKSkrKxM9OBUSps3b2YymRs2bBhiACr1+4XDTwAGz97ePiws7Pjx40qMobW1NT8/n7r+VHpXrlwRCoVDz2iqBpIaUBV8Pp/Mk1B2IAOzcOFCKyurq1evKiuAqqqq7du3MxiMAb2Kw+HweLxdu3bJKSolgnlqQCWcPHmysLCwq6vr448/Dg0NJeMDw4Wnp6cS393JyWkQr7Kzs7Ozs5N5MKoAkhpQCREREREREcqOAqgDOPwEAKgVSGoAALUCSQ0AoFYgqQEA1IoKDRSUl5eTKXxAdZDas2r5uZCLw9Vy1RSvvLycxWIpO4r/pypJjVRnBqpGdb6pMmdhYUGulgdDx2KxVOcnrCqXSQEAgEzAOTUAgFqBpAYAUCuQ1AAAagWSGgBArfwfKaZ+oK72VdAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<IPython.core.display.Image object>"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "plot_model(model2,show_shapes=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/30\n",
      "1814/1814 [==============================] - 2s 1ms/step - loss: 43.3460 - accuracy: 0.5524\n",
      "\n",
      "Epoch 00001: loss improved from inf to 43.34601, saving model to regression_model.h5\n",
      "Epoch 2/30\n",
      "1814/1814 [==============================] - 1s 336us/step - loss: 6.1098 - accuracy: 0.6279\n",
      "\n",
      "Epoch 00002: loss improved from 43.34601 to 6.10980, saving model to regression_model.h5\n",
      "Epoch 3/30\n",
      "1814/1814 [==============================] - 1s 536us/step - loss: 4.0003 - accuracy: 0.6847\n",
      "\n",
      "Epoch 00003: loss improved from 6.10980 to 4.00028, saving model to regression_model.h5\n",
      "Epoch 4/30\n",
      "1814/1814 [==============================] - 2s 862us/step - loss: 3.0619 - accuracy: 0.7089\n",
      "\n",
      "Epoch 00004: loss improved from 4.00028 to 3.06191, saving model to regression_model.h5\n",
      "Epoch 5/30\n",
      "1814/1814 [==============================] - 2s 1ms/step - loss: 2.4893 - accuracy: 0.7293\n",
      "\n",
      "Epoch 00005: loss improved from 3.06191 to 2.48933, saving model to regression_model.h5\n",
      "Epoch 6/30\n",
      "1814/1814 [==============================] - 2s 1ms/step - loss: 2.0154 - accuracy: 0.7646\n",
      "\n",
      "Epoch 00006: loss improved from 2.48933 to 2.01542, saving model to regression_model.h5\n",
      "Epoch 7/30\n",
      "1814/1814 [==============================] - 2s 1ms/step - loss: 1.7324 - accuracy: 0.7822\n",
      "\n",
      "Epoch 00007: loss improved from 2.01542 to 1.73239, saving model to regression_model.h5\n",
      "Epoch 8/30\n",
      "1814/1814 [==============================] - 2s 830us/step - loss: 1.3927 - accuracy: 0.8093\n",
      "\n",
      "Epoch 00008: loss improved from 1.73239 to 1.39271, saving model to regression_model.h5\n",
      "Epoch 9/30\n",
      "1814/1814 [==============================] - 2s 833us/step - loss: 1.3376 - accuracy: 0.8252\n",
      "\n",
      "Epoch 00009: loss improved from 1.39271 to 1.33757, saving model to regression_model.h5\n",
      "Epoch 10/30\n",
      "1814/1814 [==============================] - 2s 899us/step - loss: 1.2521 - accuracy: 0.8352\n",
      "\n",
      "Epoch 00010: loss improved from 1.33757 to 1.25213, saving model to regression_model.h5\n",
      "Epoch 11/30\n",
      "1814/1814 [==============================] - 1s 702us/step - loss: 1.1120 - accuracy: 0.8423\n",
      "\n",
      "Epoch 00011: loss improved from 1.25213 to 1.11197, saving model to regression_model.h5\n",
      "Epoch 12/30\n",
      "1814/1814 [==============================] - 1s 797us/step - loss: 0.9903 - accuracy: 0.8512\n",
      "\n",
      "Epoch 00012: loss improved from 1.11197 to 0.99028, saving model to regression_model.h5\n",
      "Epoch 13/30\n",
      "1814/1814 [==============================] - 2s 831us/step - loss: 0.8862 - accuracy: 0.8754\n",
      "\n",
      "Epoch 00013: loss improved from 0.99028 to 0.88622, saving model to regression_model.h5\n",
      "Epoch 14/30\n",
      "1814/1814 [==============================] - 1s 715us/step - loss: 0.8730 - accuracy: 0.8688\n",
      "\n",
      "Epoch 00014: loss improved from 0.88622 to 0.87299, saving model to regression_model.h5\n",
      "Epoch 15/30\n",
      "1814/1814 [==============================] - 1s 626us/step - loss: 0.8009 - accuracy: 0.8743\n",
      "\n",
      "Epoch 00015: loss improved from 0.87299 to 0.80091, saving model to regression_model.h5\n",
      "Epoch 16/30\n",
      "1814/1814 [==============================] - 1s 393us/step - loss: 0.7198 - accuracy: 0.8964\n",
      "\n",
      "Epoch 00016: loss improved from 0.80091 to 0.71976, saving model to regression_model.h5\n",
      "Epoch 17/30\n",
      "1814/1814 [==============================] - 1s 523us/step - loss: 0.6896 - accuracy: 0.8947\n",
      "\n",
      "Epoch 00017: loss improved from 0.71976 to 0.68963, saving model to regression_model.h5\n",
      "Epoch 18/30\n",
      "1814/1814 [==============================] - 1s 551us/step - loss: 0.7801 - accuracy: 0.8749\n",
      "\n",
      "Epoch 00018: loss did not improve from 0.68963\n",
      "Epoch 19/30\n",
      "1814/1814 [==============================] - 2s 911us/step - loss: 0.7525 - accuracy: 0.8903\n",
      "\n",
      "Epoch 00019: loss did not improve from 0.68963\n",
      "Epoch 20/30\n",
      "1814/1814 [==============================] - 2s 982us/step - loss: 0.7212 - accuracy: 0.9002\n",
      "\n",
      "Epoch 00020: loss did not improve from 0.68963\n",
      "Epoch 21/30\n",
      "1814/1814 [==============================] - 2s 1ms/step - loss: 0.7244 - accuracy: 0.8914\n",
      "\n",
      "Epoch 00021: loss did not improve from 0.68963\n",
      "Epoch 22/30\n",
      "1814/1814 [==============================] - 2s 858us/step - loss: 0.7574 - accuracy: 0.8853\n",
      "\n",
      "Epoch 00022: loss did not improve from 0.68963\n",
      "Epoch 23/30\n",
      "1814/1814 [==============================] - 1s 693us/step - loss: 0.7144 - accuracy: 0.8931\n",
      "\n",
      "Epoch 00023: loss did not improve from 0.68963\n",
      "Epoch 24/30\n",
      "1814/1814 [==============================] - 1s 664us/step - loss: 0.8110 - accuracy: 0.8870\n",
      "\n",
      "Epoch 00024: loss did not improve from 0.68963\n",
      "Epoch 25/30\n",
      "1814/1814 [==============================] - 2s 1ms/step - loss: 0.7297 - accuracy: 0.8925\n",
      "\n",
      "Epoch 00025: loss did not improve from 0.68963\n",
      "Epoch 26/30\n",
      "1814/1814 [==============================] - 2s 954us/step - loss: 0.5948 - accuracy: 0.9074\n",
      "\n",
      "Epoch 00026: loss improved from 0.68963 to 0.59477, saving model to regression_model.h5\n",
      "Epoch 27/30\n",
      "1814/1814 [==============================] - 1s 730us/step - loss: 0.5375 - accuracy: 0.9123\n",
      "\n",
      "Epoch 00027: loss improved from 0.59477 to 0.53747, saving model to regression_model.h5\n",
      "Epoch 28/30\n",
      "1814/1814 [==============================] - 1s 637us/step - loss: 0.5440 - accuracy: 0.9046\n",
      "\n",
      "Epoch 00028: loss did not improve from 0.53747\n",
      "Epoch 29/30\n",
      "1814/1814 [==============================] - 2s 1ms/step - loss: 0.4997 - accuracy: 0.9157\n",
      "\n",
      "Epoch 00029: loss improved from 0.53747 to 0.49968, saving model to regression_model.h5\n",
      "Epoch 30/30\n",
      "1814/1814 [==============================] - 1s 683us/step - loss: 0.4517 - accuracy: 0.9146\n",
      "\n",
      "Epoch 00030: loss improved from 0.49968 to 0.45168, saving model to regression_model.h5\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<keras.callbacks.callbacks.History at 0x7f315f920410>"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_checkpoint = keras.callbacks.ModelCheckpoint('regression_model.h5', monitor='loss',\n",
    "                                                     verbose=1, \n",
    "                                                     save_best_only=True)\n",
    "\n",
    "#-- Train model\n",
    "model2.fit(x_train.reshape(x_train.shape[0],x_train.shape[1]*x_train.shape[2]),\\\n",
    "          y_train, epochs=30, batch_size=30, callbacks=[model_checkpoint])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200/200 [==============================] - 0s 785us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[4.70385368347168, 0.6549999713897705]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#-- evaluate on test data\n",
    "model2.evaluate(x=x_test.reshape(x_test.shape[0],x_test.shape[1]*x_test.shape[2]), y=y_test, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1814/1814 [==============================] - 0s 223us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.3973665558779253, 0.9117971062660217]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#-- Also evaluate on train data\n",
    "model2.evaluate(x=x_train.reshape(x_train.shape[0],x_train.shape[1]*x_train.shape[2]), y=y_train, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the network is able to recover the regression coefficients with high accuracy for the train data and reasonable accuracy for the test data."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:mlenv]",
   "language": "python",
   "name": "conda-env-mlenv-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
